Information infrastructure management tools with extractor, secure storage, content analysis and classification and method therefor

ABSTRACT

Data processing method organizing and processing data in a distributed computing system. By organizing select/secret content with [enterprise and external designated] categorical filters (content, contextual and taxonomic) to create further search terms for data mining both enterprise and external data sources (databases, data collections, data stores). The result being aggregated select/secret content. The aggregated select/secret content is stored in the corresponding select/secret content data store. The aggregated select/secret content is further processed for convergent or divergent characteristics. By identifying unknown data elements, another search gathers supplemental documents. Data element ranges are set by taxonomic filter and said contextual filters. Relevancy factors are identified by relationship between input and supplemental documents. Search is controlled by user selection, continuous search, iterative search (n cycles), search within m search terms, and search time.

This is a regular patent application based upon and claiming thepriority of patent application Ser. No. 14/597,357, filed Jan. 15, 2015,assigned U.S. Pat. No. 11,310,274, and patent application Ser. No.11/746,440, filed May 9, 2007, now U.S. Pat. No. 9,015,301 andprovisional patent application supplemental to provisional patentapplication Ser. No. 60/883,522 filed Jan. 5, 2007, the contents ofwhich are incorporated herein by reference thereto.

A.0 INTRODUCTION

The present invention relates to information management data processingmodules and computer software system and method enabling protection ofconfidential information, identification of such information,identification of select content, classification for the same, andsecurity system for the same. Businesses and enterprises use informationin many different types of forms and format. Structured information istypically data found in database or spreadsheet formats or stored inself-defining formats wherein the information is managed by a softwareapplication. A further example of self-defining format is XML documentsin format schematics. Some information is “semi-structured” wherein theinformation has certain defined fields such as email and the structureddata fields from, to, cc, bcc and subject, but the body of the emailmessage is unstructured. Unstructured information includes documentsgenerated by end users with the assistance of applications program suchas Microsoft Office, word processing programs, image processing programs(Adobe), files downloaded from the Internet, audio and video files etc.Enterprises focus their attention on mission critical applications anddata relative thereto which is often located in structured databases orother structured data formats. However, a large majority of informationis contained in unstructured or semi-structured formats. There aresignificant risks involved in handling and locating confidentialinformation in these information files as well as identifying andhandling intellectual property data in those files.

The content of these information files are important from a legalstandpoint, a data security standpoint, and a government regulation orcompliance standpoint. Although the content of the information files maybe searched and indexed, the resulting information must then beclassified. The enterprise must establish policies which dictate how theinformation files are handled in light of legal, security and compliancerisk factors. Further, enterprise policies regarding information ordocument retention and immutability (the degree and extent that aparticular document or information file may be changed) must beestablished and enforced.

Simply indexing the content of an information file is not adequate.Semantic analysis, key word tagging and classification categorization(taxonomic analysis) should be conducted. Information tools and computerprograms and methods should enable the users (an enterprise) to analyze,categorize and actively manage unstructured information in theenterprise. Theft or accidental loss of customers' information oremployees' private information is no longer acceptable. Laws mandatedisclosure of such data loses. Further, the public is acutely aware ofidentity theft and privacy issues. Also, theft or misuse of intellectualproperty and other confidential business information is a missioncritical aspect of many enterprises. To complicate matters, businessesand enterprises do not operate closed informational systems but oftentimes invite partners, customers, vendors, contractors and other thirdparties to participate in the enterprise informational system.Therefore, most enterprises operate an open ecosystem informationalstructure. Employees of the enterprise in addition to its partners,customers, vendors and others participate in the production ofinformation and the use/and consumption of information.

In the past, most commentators believed that a breach of confidentialityand misuse of data was a result of an outsider “hacker” getting accessto enterprise information by penetrating a firewall or other securitysystem. However, there is a greater risk to the enterprise that rogueinsider will take the information and misuse that information. Further,trusted employees constantly copy files onto laptops, PDAs (personaldata assistance), USB thumb drives (flash memory), CDs and otherremovable media in order to fulfill the tasks assigned by theenterprise. This portable data can be easily misplaced or stolen.Further, the ease of electronic transfer of documents to others exposesthe information contained therein to mis-directed addresses, as well asunauthorized theft and misuse.

Therefore, enterprises and organizations must manage the access to theinformation and use of the information and monitor such access and useand audit (log) these activities. A system of information rightsmanagement should concern the following aspects: (1) control who canopen, print or edit a document or information file; (2) prevent or limitdistribution of the information file; (3) enable the enterprise tochange permissions on an information file per enterprise policy orchanged conditions; and (4) maintain an audit trail of such activity.

The information infrastructure tools also must accommodate thechangeable value of information over the life cycle of the informationfile. Although the information and content of the file does not change,the value of the information changes due to contextual events andexternal environmental conditions. External events which change thevalue of information and information file can be summarized as follows:(a) creation and modification and use of the information in theinformation file as it is processed by the enterprise; (b) completion ofthe project or event associated with the information file; (c)government regulations imposed on the information; (d) legal actionsassociated with the content of the information file and financial auditsrelated to such information; I distributing the information in theinformation file to individuals with different access or securityclearances or rights; (f) distribution of the information, bothphysically and electronically, to others who are not in the enterprise;(g) the types and operation of storage devices and the media whichretain the information files; (h) the discovery of new uses for theinformation content in the file; (i) new software applications whichreplace obsolete applications; (k) personnel changes within theenterprise and associated changes and permissions to access, control andmodify the information file; (1) expansion and contraction of theenterprise itself (organizational changes) and the changeable nature ofbusiness relationships with partners, customers and suppliers.

In view of the foregoing, it is necessary to have an informationclassification system or engine which automatically categorizesinformation in unstructured information files and labels the same andpermits the enterprise to establish a policy for that information andimplement the policy recommendation with specific actions to ensure thatthe information is properly handled, distributed, retained, deleted(document retention) and otherwise managed.

OBJECTS OF THE INVENTION

It is an object of the present invention to provide a system operator orCIO with data processing tools to manage and organize data processed byan enterprise.

It is a further object of the present invention to provide tools forsecuring secret or security sensitive sec-con data in the enterprisecomputer system and to locate, identify and secure select content SCwhich may be of interest or importance to the enterprise.

It is an additional object of the present invention to employ a dynamic,adaptive filter to enhance select content (SC) collection, to employclassification systems to organize such SC, and, in some cases, to do socontinuously.

It is a further object of the present invention to assist in dataprocessing or manipulation including processes such as coping, dataextraction, data archiving, data distribution, data destruction (adocument retention process), inter-enterprise and intra-enterprise datadistribution, data access request processing, and data distributiondenials.

SUMMARY

The present invention is a method of organizing and processing data in adistributed computing system. The invention is also implemented as acomputer program on a computer medium and as a distributed computersystem. Software modules can be configured as hardware. The method andsystem organizes select content which is important to an enterpriseoperating said distributed computing system. The select content isrepresented by one or more predetermined words, characters, images, dataelements or data objects. The computing system has a plurality of selectcontent data stores for respective ones of a plurality of enterprisedesignated categorical filters which include content-based filters,contextual filters and taxonomic classification filters, all operativelycoupled over a communications network. A data input is processed throughat least one activated categorical filter to obtain select content, andcontextually associated select content and taxonomically associatedselect content as aggregated select content. The aggregated selectcontent is stored in the corresponding select content data store. A dataprocess from the group of data processes including a copy process, adata extract process, a data archive process, a data distributionprocess and a data destruction process is associated with the activatedcategorical filter and the method and system applies the associated dataprocess to a further data input based upon a result of that further databeing processed by the activated categorical filter utilizing theaggregated select content data. In effect, the system and processtranslates the sec-con or SC data and then stores the same in certainlocations or secure stores. In this manner, a transformation of the datais achieved. By transposing the sec-con or SC data, higher levels oforganization are realized or higher levels of security are implemented.

The enterprise designated filters screen data for enterprise policiessuch as a level of service policy, customer privacy policy, supplierprivacy policy, enterprise human resource privacy policy, financial datahandling policy, public company reporting policy, health care regulationpolicy, technical innovation policy, competitive reporting policy anddocument or data retention policy.

In another embodiment, the method and system extracts security sensitivecontent from a data input to obtain extracted security sensitive datafor a corresponding security level and remainder data. This extractedsecurity sensitive data is stored in a respective security sensitiveextract store. Remainder data is stored in the distributed computersystem. Distribution of the data input with respect to said distributedcomputing system is limited based upon throughput from at least one ofthe enterprise designated filters. The controlled release ofcorresponding extracted security sensitive data from the respectiveextract stores with the associated security clearances for correspondingsecurity levels is permitted by the system. In this manner, securitysensitive content is separately stored apart from the select content,which is stored in select content data stores.

Table of Contents A. 1 Creating An Information Infrastructure 10 A. 2Sampling and Creation of a Simple Info Infrastructure 10 A. 3 SystemTools and Processing Methods for an Information Infrastructure 11 A. 4Information Dispersal - Basic 16 A. 5 Alternative to InformationDispersal Algorithm 16 A. 6 Adding Prior Art Information DispersalAlgorithms to Granular Data Dispersion 16 A. 7 Automatic Dispersal ofData to Remote Storage Locations 17 A. 8 Dispersal - Breaking Down theContext of Data Streams while Maintaining 17 Access to the Content ofits Parts A. 9 Granular Tagging - Basic 18 A. 10 Automated Granular DataLevel Manipulation 18 A. 11 Process for Creation of Automated GranularTagging and Labeling 19 A. 12 Process for Creation of Automated TaggedTear Lines 19 A. 13 Category Tags as Placeholders in Databases 20 A. 14Controlled Release - Basic 20 A. 15 Layered Build-up of Data Streams,Layer upon Layer 21 A. 16 Granular Data Release from Third PartyRepository 21 A. 17 Creation of Modular Data Components from GranularData Streams 21 A. 18 “Rolling” Exposure of Granular Data Streams 22 A.19 Controlled Release of Extracts after Verification 22 A. 20Multi-Purpose Usage of Filtering and Extraction Process 22 A. 21Filtering and Copying “Selected Data” 23 A. 22 Combined Process ofExtracting Part of the “Selected Data” and Copying Parts of the 23“Selected Data” A. 23 Data Mining Search Results Basic - Distribution asMulti Level Security Versions 23 A. 24 Data Mining - A Search CreatingAutomatically Key Words 24 A. 25 Data Mining - Continuous Data Miningwith Query Refinement 25 A. 26 Automatic Continuous Search - withDivergence and Convergence 25 A. 27 Distributed Data Mining 26 A. 28Content Inference Engine - Basic 26 A. 29 An Inference Engine -Searching for the Unknowns 26 A. 30 Securing Information Sharing with anInference Engine 26 A. 31 Automatic Web Data Inference Crawler Robots 27A. 32 Changeable Data Classifications Based on Content Inference Threats27 A. 33 Search Operations Creating New Continuously Growing DigitalData Streams 27 A. 34 Distributed Searches from Different Storage Nodes28 A. 35 Storage of Separated Data Pieces in One Location 28 A. 36Storage of Granular Data Streams - Basic 29 A. 37 Configuration ofGranular Data Streams Dispersal to Distributed Storage Stores 29 A. 38Granular Data Stream Transport 29 A. 39 Shuttling Data and Granular DataStreams Between Distributed Storage Stores 29 A. 40 Replication ofGranular Data Streams with Minimal Security Risk 30 A. 41 AnonymizingClassified and Unclassified Storage Stores 30 A. 42 Controlled Releaseof Granular Data Streams - Granular Data 30 Manipulation and Control A.43 Sanitization Process with Inference Filtering - Basic 31 A. 44Process for Sanitizing Data on the Network for Privacy Security andSurvivability 31 A. 45 Automated Sanitization of Important Data 32 A. 46Sanitization of Non-Important Data 32 A. 47 Release of Granular DataStreams in With Increasing Levels of Access Identifiers 32 A. 48 Releaseof Granular Data Streams After Obtaining Consideration 32 A. 49 SecuredRelease of Granular Data Streams into Multiple Screens 33 A. 50Ecosystem Based on “Granular Data Control” - Basic 33 A. 51 Eco Systemfor Information Superiority and Survivability Based 33 on “Granular DataControl” A. 52 System and Method for Minimizing Data Exposure 34 A. 53Creating “Digital Doors” for Access and Controlled Release in a DataStream 34 A. 54 General Aspects - Basic 35 A. 55 Automatic Creation of aUser or Object's Identification Profile 35 A. 56 Copying FilteredGranular Data and Dispersing 36 A. 57 Combining Extraction and Copyingfor Filtering of Filtered Granular Data 36 B. 1 Basic Operational Theoryof Secure or Select Data Storage 36 B. 2 General Operation 37 B. 3System Configurations 40 B. 4 Input or Initial Processing Considerations(Basic Filter Modules) 43 B. 5 Storage 44 B. 6 Extraction and StorageEnhancements 45 B. 7 Basic Reconstruction 48 B. 8 ReconstructionTechniques 49 C. 0 Testing the Security System 50 C. 1 The Etiology ofInformation 50 C. 2 Metasearch Engine for Knowledge Discovery: SearchConvergence and Divergence 50 C. 3 Initial Process - Metasearch Engineand Knowledge Expander 52 C. 4 Simple Metasearch Engine 52 C. 5Automatic Metatagging 53 C. 6 Automatic secondary tagging of a “range”53 C. 7 MLS, Multi-level Security-Compliant Search and Distributions 54C. 8 Benefits of the Metasearch - Knowledge Expander 55 C. 9 InformationLife Cycle Engine 56 C. 10 Information Life Cycle Searches 59 C. 11Search With Prioritized Classifications 61 C. 12 Simple Filters 62 C. 13Complex Filters 62 C. 14 The Editor As An Information Expander 64 C. 15Document Object Model (DOM) - Protection and Processing 67 C. 16Recognizing The Document Object Model (DOM) 70 C. 17 Content 72 C. 18DOM Process Editor 74 C. 19 Five Informational Attributes 74 C. 20Select Content Introns and Exons 75 C. 21 Hierarchical Taxonomic ClassExamples 78 C. 22 Knowledge Expander (KE) Basic Program 79 C. 23 SecuredMetasearch Engine Based on Granular Data Control 81 D. 1 Secure Editor,DOM Editor (General) and DOM Editor Applied as Extractor Modules 85 D. 2Applications of the Secure and SC Data Systems 87 D. 3 AutomaticFeatures 87 D. 4 Multiple Security Levels 88 D. 5 Sharing Data withDifferent Security Levels - Data Mining 91 D. 6 Multilevel Securitythrough Sanitization with Reconstruction of Sanitized Content 93 D. 7Multiple Independent Levels of Security (MILS) 97 D. 8 MLS Server andWorkstation Components 100 D. 9 Flexibility of the Present MLS Approach100 D. 10 Multiple Extraction Filters and Application Outline 102 D. 11Multiple Security Features for Data 103 D. 12 Adaptive Data Security-Event Responsive System 107 D. 13 Email, Web-based and Other Types ofApplications 109 D. 14 Portable Computing Device Environment 113 E. 1General System Comments 116

BRIEF DESCRIPTION OF THE DRAWINGS

Further objects and advantages of the present invention can be found inthe detailed description of the preferred embodiments when taken inconjunction with the accompanying drawings in which:

FIGS. 1 a and 1 b diagrammatically illustrates a basic system diagram,method and flow chart for an information infrastructure.

FIG. 2 diagrammatically illustrates a basic information life cycleprocess or system wherein secure content (sec-con) and select content(SC) is separately stored per categorical filters operable by theenterprise.

FIG. 3 diagrammatically illustrates a simplistic dynamic filter andselect content (SC) storage coupled to a data processing event.

FIG. 4 diagrammatically illustrates a basic system diagram showingfiltering and storing extracted data and remainder or common data and,in an enhanced embodiment, generating and storing a map.

FIG. 5 diagrammatically illustrates a system diagram showingreconstruction of the data, various security clearances and bothelectronic reconstruction and visual reconstruction.

FIG. 6 diagrammatically illustrates a system showing major components ofa single personal computer (PC) system, a networked system with severalPCs (a LAN or WAN) and the network coupled to a telecommunicationssystem and the Internet and shows the interconnection with a server andmultiple, Internet-connected memory units.

FIG. 7 diagrammatically illustrates a basic flowchart showingreconstruction for various security levels.

FIG. 8 diagrammatically illustrates interleaving distinct data intodifferent memory locations in a video memory.

FIG. 9 diagrammatically illustrates a flowchart showing one embodimentof the principal portions of the data security program.

FIG. 10 diagrammatically illustrates a flowchart showing the basicelements of the reconstruction process for the data security program.

FIG. 11 diagrammatically illustrates deconstruction of an input documentinto segments.

FIG. 12 a diagrammatically illustrates the information life cycleknowledge expander (KE) search engine process.

FIG. 12 b diagrammatically illustrates the KE process of using searchterms, data collection and depth-of-search control.

FIG. 12 c diagrammatically illustrates the KE process for the primarysearch results.

FIG. 12 d diagrammatically illustrates the KE process for the outputrepresentation of search term data and search result data ofsupplemental documents.

FIG. 13 diagrammatically illustrates a filter program.

FIG. 14 diagrammatically illustrates an editor program.

FIGS. 15, 16, 17 and 18 diagrammatically illustrate extraction of selectcontent (SC) and a hierarchical taxonomic system based upon TS, T, C andUC security code classifications.

FIG. 19 diagrammatically illustrates a data object tree or map for adocument model (DOM).

FIG. 20 diagrammatically illustrates varies metadata levels in the DOM.

FIG. 21 diagrammatically illustrates a DOM editor process.

FIG. 22 diagrammatically illustrates another DOM editor process.

FIGS. 23, 24, 25 show examples of categorical identifiers, ranked idcategories and categorical ids for a business.

FIG. 26 diagrammatically illustrates a basic KE search process.

FIG. 27 diagrammatically illustrates an expanded KE search process.

FIG. 28 diagrammatically illustrates KE search process options.

FIG. 29 diagrammatically illustrates a hardware implementation for theKE search system.

FIG. 30 is a computer network diagram showing various user communities.

FIGS. 31 a and 13 b diagrammatically illustrates a flowchart showing thekey component steps for the multiple layer security program for thecommunity of users.

FIG. 32 diagrammatically illustrates a multiple level security systemaccessed by users having different security clearances (which alsorepresents a data mining system and operation).

FIGS. 33 a and 33 b diagrammatically illustrates a flowchart showing thekey components of an adaptive security program adaptable to variouslevels of electronic attacks, hacker or hack attacks.

FIGS. 34 to 39 show tabular examples of attack and response levels forStandard Automatic Defenses Matrix; Optional Automatic Defenses Matrix;Security Meter Module; Normal Work Mode; Threat Made Table; and, AttackMode.

FIG. 40 diagrammatically illustrates a flowchart showing the keycomponents of a multiple encryption program using multiple types ofencryption in one document or data object. Encryption may include datacompression and/or expansion or parsing techniques.

FIG. 41 diagrammatically illustrates a chart showing the key componentsof the parsing, dispersion, multiple storage and reconstruction (undersecurity clearance) of data.

FIGS. 42 and 43 diagrammatically illustrate a flowchart showing the keycomponents of one embodiment of the e-mail security system (jump pointsD-11-A and D-11-B link the flow charts).

FIGS. 44 a and 44 b diagrammatically illustrate a flowchart showing thekey components of one embodiment of the invention implements thesecurity system on a web browser (jump point 12-A links the flowcharts).

FIG. 45 diagrammatically shows several revenue systems which may beemployed with the data security systems described herein.

FIG. 46 diagrammatically illustrates a portable computing device (or thecritical parts thereof, see FIG. 6 for further details) associated withthe portable data security locator system and method.

FIG. 47 diagrammatically illustrates a basic flow chart for the portablesecurity system program in accordance with the basic principles of thepresent invention.

FIG. 48 diagrammatically illustrates a MILS or multiple independentlevels of security computer network with three (3) sub-networks atsecurity levels top secret TS, secret S, and unclassified U.

FIG. 49 diagrammatically illustrates a security guard computer useful inthe network of FIG. 16 .

FIG. 50 diagrammatically illustrates a workstation useful in the MILSnetwork.

FIG. 51 diagrammatically illustrates a filter program flow chart.

FIG. 52 diagrammatically illustrates a basic security editor program.

FIG. 53 diagrammatically illustrates a general flow chart for acomprehensive filter operable on a document DOM.

FIG. 54 diagrammatically illustrates a basic application of thecomprehensive DOM editor.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention relates to an information infrastructure with manydifferent features and characteristics. The infrastructure is a system,a methodology and a process implemented on a personal computer (PC)system, on a computer network (LAN or WAN) and over the Internet andcomputer programs and computer modules and an information processingsystem to accomplish the functions described herein. An AbbreviationsTable is provided at the end of this specification. Also, the GeneralSystem Comments section E.1 describes certain aspects relative to theimplementation of this invention. In the drawings, and sometimes in thespecification, reference is made to certain abbreviations. TheAbbreviations Table (section E.1) near the end of the specificationprovides some correspondence between the abbreviations and the item orfeature.

A.1 Creating an Information Infrastructure

With respect to unstructured data (including semi-structured data) ininformation files, the prime considerations in creating an informationinfrastructure involve a recognition of different service levels,different retention-destruction periods for the files and differentprotective measures imposed by legal, security and compliancerequirements. Confidential data may involve the enterprise, itsemployees, its customers and/or third parties. Further, the enterpriseshould gather, control disseminate and utilize its intellectual property(IP) rights in and to the data in unstructured data files. The currentmind set relative to an open informational ecosystem, wherein employeesof an enterprise, at many levels, openly communicate with enterprisepartners, customers, vendors, contractors and other third parties,results in many communications paths wherein confidential data and IPdata is subject to dispersal, misuse and theft. Concern over hackersaccessing such sensitive data, rogue insiders taking and misusing thedata and accidental loss of data by authorized users requires that theenterprise build an information infrastructure to identify, contain andclassify the confidential data and IP data.

Information Rights Management (IR Mgt) can and is presently imposed ondata files with the use of data file metadata, tags or labels. Forexample, the file property may be set at “Read Only” which limitsrevision rights. However, the content in information files is notsubject to many controls. IR Mgt (a) controls the open, print and editfunctions; (b) limits access, permissions, and distribution; (c)establishes and permits changes to permission levels; and, (d) providesand audit trail. Since the value of the information changes over thelife cycle of the information file, oftentimes due to events and objectsoutside of the information file itself, it is the content that should bemanaged and subjected to an information infrastructure.

The present invention employs an extraction engine which identifies,selects and then extracts the security sensitive content (sec-con) orselect content (SC), a classification engine that assists the user (theenterprise's CIO) to classify and organize the content, and a vibranttesting cycle to continually challenge the sec-con store, the sec-consecurity levels and the quality, breadth, depth and scope of the SCstore. The classification engines can be used by the Chief InformationOfficer (“CIO”) to establish information infrastructure policies andre-set the extraction engines to enforce the policies (data acquisitionand handling rules) established thereby.

A.2 SAMPLING AND CREATION OF A SIMPLE INFO INFRASTRUCTURE

As discussed above, mission critical data is oftentimes captured instructured data compilations such as databases and spreadsheets. Herein,a “spreadsheet” is included in the definition of a database because aspreadsheet is a two-dimensional database. Therefore, confidential datain unstructured information files which matches some or all of the datain structure files represents the initial starting point to definemission critical enterprise data. Some of this some of this missioncritical enterprise data represents secret or confidential data ofvarious parties (the enterprise, its partners, customers, vendors,contractors and other third parties) and IP data of various parties.Data from the structured files initially can be used as a content filterto extract the same confidential information from unstructured files andstore the security sensitive sec-con data in extract stores as explainedlater herein. A simple classification system (hierarchical taxonomicsystem) can be established by reviewing the label descriptions on thestructured data and then expanding class definitions with the use of theknowledge expander KE search engine described later in section C.2 andC.22, among others. The data and information returned from the KE enginecan be used by the CIO (system administrator) to create increasinglybroader or higher levels of data classifications. The hierarchicaltaxonomic system can be used to build contextual filters and taxonomicfilters which can further protect sec-con data and expand the value andquantity of SC data. Once the CIO has established the classes of data,he or she can set policies for (a) storage and retention; (b)extraction-dispersal-access-release; and (c) levels of service andpermission levels.

After removal and secure storage of known security sensitive data fromunstructured data sources (using filters designed from structured datasources), a review of unknown data in unstructured files may beappropriate.

A.3 SYSTEM TOOLS AND PROCESSING METHODS FOR AN INFORMATIONINFRASTRUCTURE

FIGS. 1 a and 1 b diagrammatically illustrate a basic system diagram,method and flow chart for an information infrastructure. Theinfrastructure is a dynamic process and hence the CIO or systemadministrator can enter the process at many points. Information (Info)infrastructure 10 includes a number of modules or process routines orfunctions which are described in detail in sections B.1-D.14. The primeoperational modules or functions are set forth in FIGS. 1 a and 1 b(which is a composite of FIGS. 1 a and 1 b ). The prime objective of theinfo infrastructure is to process data from source document or sourceinformation file 12. The source is first processed via an editor module14 to deconstruct the source doc into its constituent parts. Seesections C.14-C.22, and more critically, section D.1.

After deconstruction (which may be optional per the system operator'schoice), the infrastructure processes data with extraction module 15.Extracted data and remainder data is processed by secure storage Sec-Conmodule 16 and, if appropriate to the CIO's designed task, the selectcontent SC storage module 17. The extraction and secure store isdiscussed throughout the specification but is particularly discussed insections B.1-B.6. The select content SC store module is discussed insections C.1-C.13. C.20-C. 22. The use of an editor module in sectionsC.14-C.19 is used to process input data which, after such processing, isapplied to the content filter modules, the context filter modules andthe taxonomic filter modules. The system coordinates storage of datawith map module 18.

Extraction module 15 handles input from content filter module 20,contextual filter module 21 and taxonomic filter module 22. Theinfrastructure system and process is adaptive therefore the extractionmodule also outputs data to filter modules 20, 21 and 22. Contentfilters are discussed in many places in the specification (seeparticularly sections B.4, and C.12). Since the extraction is employedwith the (a) security system and (b) the knowledge expander KE system togenerate select content SC data, this dual purpose both feeds data tothe filters modules 20, 21 and 22 and accepts data therefrom to definethe filters used to extract critical data from the data flow. Acontextual filter module 21 is employed in an infrastructure securitysystem in order to assure the CIO-system operator (“sys-admin”) that (a)the source document (source doc) is properly scrubbed and (b) to add newwords, terms or data objects to the list of security sensitive dataobjects of concern. See sections B.4, C.13, C.20. The new data objectsbecome content filters for another processing cycle for a new sourcedoc. When configured as a knowledge expander KE infrastructure forselect content SC, contextual filters increase the SC data extractedfrom the source doc and increase the value and informational content ofthe SC store. The taxonomic filter (“tax-filter”) permits the sys-adminto employ a higher conceptual layer on either the security extraction orthe SC extraction and compilation. Tax-filters are discussed in sectionsC.1-C.22 and particularly in sections C.1, C.11, C.14, C.21 and C.22.

The sys-admin may use the info infrastructure with an existingclassification system and information policy, ruled-based or may have nopre-existing classification system for his or her documents. Theinfrastructure process 10 includes a classification generator module 24which employs an SC extractor, a KE search engine to generate additionalsupplemental documents (supple docs) which the sys-admin may use thecreate the hierarchical taxonomic classification system. otherwise, thesys-admin may purchase taxonomic classifications commercially fromothers. However, the sys-admin must ultimately prioritize thehierarchical taxonomic for the enterprise. The taxonomic system can becreated by operation of the modules in sections C.1 through C.23. The KEexpander module can be configured to list docs and data entries and thislisting can be used by the sys-admin to (a) build the hierarchicaltaxonomic and (b) prioritize the classes. Once the hierarchicaltaxonomic classes are gathered, the sys-admin defines which tree, branchand leaf is critical in module 26, then defines (or redefines) securitylevels, sets secure stores, ids SC, sets SC stores and sets permissionlevels, service levels, access levels and other policies for theenterprise which policies are translated into a rules-based system forthe processing of data. Module 32 applies the rules 32A, 32B, 32C, and32D to filter modules 20, 21 and 22 and to the access-permission moduleor control function 42. Data requests 40 from others are input intoaccess, permission, distribution, reconstruction control module 42.Features and functions for this module are discussed in sections B.7,B.8. C.9-C.14, C.20-C.23 and in the applications D.2 through D.14. Therequest for data may be an alarm function or may be a data request formanother. The info infrastructure then cycles back to source doc 12,editor module 14 and extractor module 15.

To test the security levels and system, the sys-admin may activate testmodule 28. Section C.0 discusses how the KE search engine is employed totest the security system.

FIG. 2 diagrammatically illustrates a basic information life cycleprocess or system wherein secure content (sec-con) and select content(SC) is separately stored per categorical filters operable by theenterprise. Data lifecycle 1 in general begins with an acquisition 4 andprocess or storage 5 a in the enterprise distributed computing system orenvironment represented by data lifecycle ring 3 a and outer ring 3 b.Storage memory units 6 a-8 a are part of the distributed computer systemof the enterprise. However, the enterprise may contract out the storagefacilities. In any event, the enterprise controls storage 6 a-8 a. Somestorage units 7 a-8 a may be combined. However, archive storage 8 a maybe less expensive (slower process and access time) than the SC storageunits 7 a-7 e. Further, the filter SC storage units 7 a-7 e may be in asingle memory, and simply segmented by control parameters. In general,as explained in sections B.1-B.8 and others (see D.7-D.9), the memoryunits 6 a, 6 b and 6 c are distinct and access to these sec-con storesis controlled with software or hardware guards (see section A.23 or D.6)or the extract stores are physically separate. Some data stores may becopy protected.

Data on the right side of inner enterprise ring 3 a is in transit 2 adue to acquisition 4 and other high frequency access, distribution andprocessing aspects (service levels). A high service level may beassigned to data in transit 2 a. Data in process section 2 b isundergoing manipulation and processing by the enterprise. This mayinclude the use of sec-con filters and extraction to top secret TS store6 a, secret S store 6 b and classified C store 6 c. Remainder data iskept in the enterprise store in ring 3 a.

Enterprise designated categorical filters are used to store selectcontent SC relative to the category in certain SC stores. Thesecategories cover a wide range of topics which may be of interest to theenterprise. For example, enterprise designated filters screen data inprocess 2 b for various enterprise policies including, but not limitedto level of service policies, customer privacy policies, supplierprivacy and proprietary informational policies (business privacyissues), enterprise human resource privacy policies, financial datahandling policies (see GLB or Gramm-Leach-Bliley Act of 1999), publiccompany reporting policies (see SOX or Sarbanes Oxley Act or PublicCompany Accounting and Reform and Investor Protection Act of 2002),health care regulatory policies (see HIPPA or Health InsurancePortability and Accountability Act), technical innovation policies(intellectual property (IP) disclosures, patent disclosure systems,trade secret tracking systems), competitive reporting policies (acollection of all data on the enterprise's prime competitor) anddocument or data retention policies. The enterprise sets these policiesand the CIO or sys-operator fashions filters which either continually orupon certain events or periodically sweep the enterprise data system,locate SC and store the SC in the relevant data bases. As a furtherexample, technical information on a certain product may be importantand, in light of the fact that 60-80% of all corporate data is inunstructured documents, the data lifecycle sweep may filter allprocessed data looking for that certain product. The technical filtercontent file is expanded to include additional contextual materials bylooking at outside or extra-enterprise data, thereby creating aggregatedSC data for the special technical data SC filter. With the applicationof a taxonomic classification system, the product specific tech filteris further expanded to cover higher classifications of product types andlower, more detailed products that relate to the target product. Across-check operation with IP SC filter and database 7 e also providesguidance. If the IP is very valuable, it may be stored in sec-conextract stores 6 a-6 c.

FIG. 3 diagrammatically illustrates a simplistic dynamic filter andselect content (SC) storage coupled to a data processing event. SCfilter 45 may be one or more of a content-based filter, contextualfilter or a taxonomic classification filter. The SC filter 45 accepts,in a first instance, data input 1A and, in a second instance, furtherdata input 1B. As explained below, data 1A may be the same as data 1B.As an example, a sequential operation is described. Input data 1A is fedto SC filter 45. The SC filter identifies SC content SC-1A and thiscontent is sent to SC database 47. Any type of storage may be usedrather than a database, however, some type of indexing is useful. The SCcontent copied from data input 1A is added “cc” to SC filter aggregated48. The SC database uses the SC aggregated data to supplement the SCfilter 45 with a summation+function via operational path 49.

The next process supplies further data input 1B to SC filter 45. Thedata 1B is processed by the filter 45 and select content SC-1B is copied“cc” to SC filter aggregation file 48 and also copied “cc” to the SCdata file or structure 46. The SC aggregate is fed back to the SC filtervia route 49. In this manner, the SC filter can be one or morecontent-based filters, contextual filters or taxonomic classificationfilters. This is the adaptive and dynamic filter for the categoricalfiltering discussed above. If data 1A is obtained outside of theenterprise, the SC filter expands. See KE expander system in sectionsC.2-C.22. Outside or external data is primarily to expand the filter. Itis stored in the SC aggregate file. If the input data 1A is internal orintra-enterprise data, both the SC aggregate file is expanded and thesys-operator needs to know that the SC is an internal document and thisinternal SC-1B is stored in SC data file 46.

Data 1B is also applied to control 52. The SC filter 45 generates aresult command 51 or an output control signal to indicate the presenceor absence of certain SC data in data block 1B. Contextual relevance ortaxonomic key words or terms may also trigger filter result output 51.This signal is applied to control 52 which activates data process 54.Some data processes are copy, extract, distribute, archive, or destroy.Further, as discussed below, SC filter 45 may be set as a screeningfilter which is then used to identify a certain categorical filter whichthen triggers the data process. Alternatively, the data input 1A or 1Bmay come with a data process request and the data must be filtered orscrubbed before the data process is carried out by the enterprisecomputer system.

Some key operations of the system and process follow. The inventionorganizes and processes data important to an enterprise operating thedistributed computing system. The SC or select content is represented byone or more predetermined words, characters, images, data elements ordata objects. The computing system has a plurality of select contentdata stores 7 a-7 e for respective ones of a plurality of enterprisedesignated categorical filters which include content-based filters,contextual filters and taxonomic classification filters, all operativelycoupled over a communications network. An archive filter 8 a operateswith data at rest 2 c at the top left of lifecycle 1. Aged content issubject to archiving policies or back-up policies. A data input 1A or 1Bis processed through at least one activated categorical SC filter 45 toobtain select content, and contextually associated select content andtaxonomically associated select content (SC-1 a) as aggregated selectcontent. The aggregated select content is stored in the correspondingselect content data store 47 and, more specifically, filter aggregatefile 48. A data process 54 from the group of data processes including acopy process, a data extract process, a data archive process, a datadistribution process and a data destruction process is associated withthe activated categorical SC filter 45 and the method and system appliesthe associated data process 54 to a further data input 1B based upon aresult 51 of that further data 1B being processed by the activatedcategorical SC filter 45 utilizing the aggregated select content dataper path 49.

In another embodiment, the method and system extracts security sensitivecontent TS or S or C (FIG. 2 ) from a data input to obtain extractedsecurity sensitive data for a corresponding security level and remainderdata. Remainder data can be considered left-over data which isextraneous to the sec-con or SC. It may be surplus or residue data. Inthis manner, the remainder data is complementary to the sec-con or SCdata. This extracted security sensitive data is stored for thecorresponding security level in a respective security sensitive extractstore 6 a, 6 b or 6 c. Remainder data is stored in the distributedcomputer system. Distribution of the data input 1A (FIG. 3 ) withrespect to said distributed computing system is limited based uponthroughput from at least one of the enterprise designated SC filter 45(e.g., patient privacy or HIPPA filter linked to SC store 76). Thecontrolled release of corresponding extracted security sensitive datafrom the respective extract TS or S or C stores 6 a-6 c with theassociated security clearances for corresponding security levels ispermitted by the system. In this manner, security sensitive content isseparately stored apart from the select content, which is stored inselect content data stores 7 a-7 e and 8 a.

The sys-operator may activate ON a categorical filter or the system maybe automatic. Prior to activating a designated categorical filter (seeGLB SC content filter 7 c, FIG. 2 ), the data input 1B is processedthrough a screening taxonomical classification SC filter 45 which output51 activates said at least one designated categorical filter for thesensed data input 1B (FIG. 3 ). Several categorical filters may beactivated, operating concurrently or sequentially, dependent uponprocessing speed and memory access, based upon the screening taxonomicalclassification filter output.

The further data input 1B may be the same as the initial data input 1A.In other words, an iterative process may be imposed on data 1A/1B.First, the SC filter 45 is used to expand the SC content, maybe to addcontextual material, or to add a higher or lower classification SC dataper the taxonomic filter. Thereafter, the data 1A/1B is processed againby SC filter 45 with the data process 54 associated therewith.

Regarding the initial data processing, the further data input 1B may bestructured or unstructured data. If structured, the data input should bedeconstructed and special attention should be paid to the structureditems. In email, a partially structured data item, addressee and theaddressor data and the regarding field often are key SC items for theenterprise. If unstructured data is filtered, a deconstruction processshould be used. See sections C.14 and D.1, among others. The constituentdata elements are then SC filtered by 45.

Activating the enterprise designated categorical SC filter 45 may beautomatic or manual. An automatic activation can be is time-based (2:00AM every day), distributed computer system condition-based (when dataflow falls below x % or process speeds exceed y %), or event-based (seesection D.3, D.11, and others which discuss event triggers). Activationof the categorical filter can be linked to a data process request fromthe group of data processes.

The data processes include a copy process, a data extract process, adata archive process, a data distribution process and a data destructionprocess, an inter-enterprise data distribution process, anintra-enterprise data distribution process, a data access requestcoupled with said data distribution process (a request to send data 1Ato a newspaper via email (copy and distribute)), a data distributiondenied process, and a limit data distribution process. The data processassociated with a data block 1A/1B may trigger one or more SC filters ora discovery of SC in a data block 1B may cause a change in the dataprocess from “full distribute” to limit distribution to key enterprisepersons.” In litigation, discussions of the lawsuit may be an SC filtertopic. The groups of data processes may activate different categoricalfilters. The result 51 of the further data 1B processed by the activatedcategorical SC filter 45 indicates the presence or absence of selectcontent or aggregated select content in the data 1B.

On the output side, the system may impose a classification taxonomy onthe stored aggregated select content SC file 48 (FIG. 3 ). This enablesthe sys-op to see the scope of the filter (depth, detail, higher levelclassifications, orthogonal data fields, etc.). See, for example,section C.22. Preferably, the select content file 46 is stored separatefrom the aggregated select content 48 in the select content store 47. Aseparate storage permits the sys-op to see what SC data resides in hisor her enterprise as compared with the scope of the filter. The systemmay impose a classification taxonomy onto the stored select content 46.See FIG. 12 a . Further, the stored SC 46 may be a copy of the obtainedselect content from data 1A/1B or may be a representation thereof insaid select content data store. A representation may be a marker orpointer to an index of SC. This process avoids multiple copies of SCstorage. Additionally, in order to track the origin of the data 1A/1B,the data input has a source indicator (like a URL or the path to theoriginal document location) and the obtained select content SC-1A orSC-1B, when stored in the select content data store 46, includes acorresponding representative source indicator pointing to or linking tothe source of the data. In this manner, the sys-op knows where the SCdata came from. Additionally, the process may generate a tag or labelresponsive to the source indicator of data 1A, and the correspondingrepresentative source indicator. The tag or label can be electronicallyattached to the source document to indicate SC content, importance ofthe SC content and category thereof.

A.4 INFORMATION DISPERSAL—BASIC

The dispersal of information into the secure storage or the selectstorage, with maps to the original or source document, is one of severalimportant aspects of the information infrastructure. The storageprocesses and modules provide an alternative to prior art informationdispersal algorithms. A prior art information dispersal algorithm isdiscussed in U.S. Pat. No. 5,485,474 to Rabin and in an article entitled“Survivable Information Storage Systems” by J. Wylie M. Bigrigg, J.Strunk, G. Ganger, H. Kiliccote, and P. Khosla, published August, 2000in COMPUTER, pp. 61-67. The present system of information dispersal canbe additive to the prior art information dispersal algorithms. Thepresent system and method can be automated to facilitate dispersal ofinformation to remote storages upon detection of certain events. Lastly,the information dispersal techniques can provide for the dispersal ofcontext of the whole data stream while maintaining access to theconstituent parts of the content.

A.5 ALTERNATIVE TO INFORMATION DISPERSAL ALGORITHM

The invention provides a system and method for dispersal of plain textdata and an alternative to usage of prior art information dispersalalgorithms. The usage of prior art information dispersal algorithms doesnot permit the user or sys-admin to view or use data stored within aparticular storage. The prior art algorithms permit one to view and haveaccess to data only when data is retrieved from a few stores and then iscombined together.

The present inventive process and system enables use of granular datawhich is which is stored in a known and accessible storage store (indefined storage locations (sec-con store or SC store). The security riskin the present system can be reduced by storing smaller and moregranular pieces of data. Further, after splitting granular data from asource document or data stream, the granular pieces of data may beprocessed using the prior art information dispersal algorithms.

A.6 ADDING PRIOR ART INFORMATION DISPERSAL ALGORITHMS TO GRANULAR DATADISPERSION

The present invention provides a system and method for dispersal ofgranular data to distributed storage stores and the possibility to addthe prior art Information Dispersal Algorithms (“IDA”). The inventionprovides a system and method for dispersal of data to distributedstorage stores—in essence an inherent “built in” security feature. TheInformation Dispersal Algorithm or IDA delivers a relatively high levelof security; however, it does not permit nor allow access to datacontent for search and other Knowledge Management operations.

On the other hand, the present invention may (a) first granularlyextract the data form the source, then (b) use the IDA for securingextracted granular pieces of data, one at a time. This combinationbrings flexibility to the system as a whole since granular pieces can bereconstituted, one at a time, and released from IDA for knowledgemanagement operations without compromising the security of the wholedocument. By securing granular data pieces with the InformationDispersal Algorithm or IDA, the system's granular data parts oncereconstituted by the IDA are available in system storage and arestand-alone data structures (encrypted or not). These stand-alone datastructures and the granular data therein can be read on their ownwithout the need to bring together other data shares. Because extractscan be in plain text or decrypted and stand in their own data structure,the sys-admin can authorize an advanced search and knowledge managementoperations through the granular data structure.

Splitting a data stream and dispersing the slices of the split data todistributed data stores creates a very high level of security. Thisemulates military tactics of splitting and hiding forces. In words ofthe great strategist, Sun Tzu, the present process and system creates“disorder from order” (the order being the source document, the disorderbeing the granular and dispersed data parts) and it creates a formlessorganization which makes it a very hard target for the attacker. Thelevel of security may be further raised if, at the storage stores, thereare strong access control mechanisms for authentication and validationof users. Dispersal of data to distributed storage stores achieveshigher security as well as a lower overall security costs.

Distributed storage stores need less security than a centralized datarepository for a number of reasons. First, the distributed storagestores hold only parts of the data and they are of lower interest to anattacker that will need to attack few dispersed stores to get the totalcontent. Second, the stores are scattered and if hidden they call forless security. The need for less security means lower costs; moreefficiency and less processing power. Thus, dispersal of data todistributed storage stores is inherently “built in”, “baked in”security. The split granular data pieces may be sent to distributedstorage to be processed in an Information Dispersal Algorithm scheme.Each granular item will be split into n pieces, which will be sent todistributed storage. Reconstructed of a granular item will be enabled byaccess to M out of N pieces. A directory/map connects all granularpieces recreate a partial or original document. Using the InformationDispersal Algorithm (IDA) with the present invention may deliver a muchhigher degree of security. The data stroes may be correlated to thesec-con or SC content or maybe otherwise designated.

A.7 AUTOMATIC DISPERSAL OF DATA TO REMOTE STORAGE LOCATIONS

The present invention provides a system and method for the automaticdispersal of data to remote storage locations. Examples of these systemsand methods are found in sections D.2 through D.14. Any document datastream created by a user may be automatically split and transported todistributed storage locations. This may take several forms. One form is,when a file is being closed, it will automatically be split and itspieces sent automatically to distributed local or remote storagelocations.

A.8 DISPERSAL—BREAKING DOWN THE CONTEXT OF DATA STREAMS WHILEMAINTAINING ACCESS TO THE CONTENT OF ITS PARTS

The present invention provides a system and method for splitting adocument or data stream into granular parts for security and privacyreasons which hides the data stream's context. By splitting a documentor data stream into different granular pieces and dispersing the piecesinto distributed storage, the context of the original document or datastream is broken. The more splits and slices imposed on the data, theharder it is to inference the content, concept and context of the sourcedocument or data stream. The uniqueness of this invention is that eachdistributed piece and its granular content can be independently accessedfor different usage. This storage technique is very different than theRabin Information Dispersal Algorithm (U.S. Pat. No. 5,485,474) in whichslices of the original document are distributed in different storagelocations but the content of each slice or piece of data is notaccessible.

The present system and method permits the sys-admin to establish an infoinfrastructure for data mining a plurality of source documents orinformation files. See sections C.8, C.17, D.5 and D.13. The sys-admincan also employ the knowledge expander KE search engine to enhance thedata mined form source docs. See sections C.2 through C.23.

A.9 GRANULAR TAGGING—BASIC

The present method and system permits granular extraction and tagging ofdata. As a result, the system permits (a) automated granular data levelmanipulation; (b) a process for creation of automated granular taggingand labeling; (c) a process for creation of automated tagged tear lines;and (d) use and implementation of category tags as placeholders indatabases.

A.10 AUTOMATED GRANULAR DATA LEVEL MANIPULATION

The system and method provides for granular data manipulation. All inputdata is analyzed, optionally categorized and optionally presented in amatrix or other format (a data structure). The analysis involvesdeconstruction or decompilation of the source doc (see section C.14-C.16 and, more particularly, section D.1. Categorization is discussed insections C.11 and C.21. Different tools enable selection of granulardata items for different activities which may include, among otherthings, their use as: (1) key words to search engines; (2) headings forgranular data elements, meta tags for such elements; (3) a base forbuild up of expansive content (supple docs); and (4) filling in missinginformation from documents (data mining third party sources.

Granular data control may be used for various information processingactivities listed in the Action Table below. The Action Table, in oneembodiment of the invention, is a pull-down menu which enables the userof the system to select the granular action associated with theactivity.

Action Table (1) security (2) privacy (3) compliance (4) archiving (5)replication (6) backup (7) continuity (8) survivability (9) broadcasting(10) alerting

The granular actions are selected automatically (based upon the initialset-up of the system), manually or a combination of both. Granular itemsmay be connected to other granular items in the matrix/format orconnected to other data processing steps external to it. Different datamatrixes or data structures in the sec-con store or SC store can beconstructed for the granular output of the original, source input data.The original data ad constituent parts thereof may be replicated to formdifferent matrixes based on different categorizations. The selectedgranular data items will usually have semantic meaning. However, it isfeasible to combine granular items in different ways which do not havean inherent semantic meaning. This might be done for search and contentinference purposes. By categorizing and labeling every granular dataelement, the invention in essence differently weights different granularcontent items.

A.11 PROCESS FOR CREATION OF AUTOMATED GRANULAR TAGGING AND LABELING

The invention provides a system and method for automatic creation oftagging and labeling at the granular data level. See sections C.3, C.5,C.6, C.7 and D.1. There is a need for automation in tagging data andcontent in a document and data stream. the tagging and labeling can beused for security or for data mining or knowledge expansion. The needsecurity tagging includes: Top Secret paragraph, Secret paragraph. TopSecret line, Secret line (tagging plus contextual filter set as a range(paragraph and line). There is a need for tagging at a more granularlevel such as Top secret word/character, or Secret word/character. Theneed for tagging at a very granular level also is impressed upon anenterprise by the requirements of recent laws for removal of sensitivenames and sources from documents. There is a need for other types ofcategorization/labeling/tagging. For example, there is a need for aPrivate tag, Public tag, and a need for High Priority, Low Priority,Medium Priority tags. The enterprise policy discussed above, whenapplied with the use of the present method and system, implements thiscategorization/labeling/tagging operation. Each item, word, character,image in a data stream (also optionally at a bit and binary level) ismonitored, analyzed, categorized and tagged. This is done by filtering,i.e., matching the granular data items against pre-set lists wherein amatch yields the right classification tag for the granular data element.The element is marked by creation of a tag for the granular dataelement. For example, “name,” “location,” and “social security number.”

Following process outlines an automatic tagging system. (1) Within theinput data stream, all the data and content element is monitored andcategorized by a filter or filters. The filters may be content filters,like a “dirty word” list of others. (2) Every data element iscategorized contextually (name location, phone etc). Every element maybe also labeled, tagged, and categorized according to othercategorizations like security classification, Top Secret. See, forexample, the discussion of hierarchical taxonomic in section C.21. (3)Every tag may be checked contextual validity and may be checked andverified by an inference engine.

By categorizing and labeling every granular data element, the inventionin essence different weights to the different granular content items.

A.12 PROCESS FOR CREATION OF AUTOMATED TAGGED TEAR LINES

The invention provides a system and method for automatic creation ofautomated tear tagged lines or contextual ranges about identifiedcontent. There is a need for automation in classifying ranges of contentwithin a document or data stream. The need may be for security rangeslike Top Secret paragraph, Secret paragraph etc. as well as for othertypes of categorization/labeling/tagging. For example, there may be aneed for Private tag, Public tag, High, Low or Medium priority tags.Based on automatic categorization of each item, word, character, image,in a data stream, a contextual defined range, such as a line a paragraphor a group of words, can be tagged by one of the tags in that grouping.For example, if a paragraph has 15 words with 15 tags, the paragraph maybe tagged as a whole based on one or few of the tags. In situationswhere tags represent a hierarchy—(“Top secret, Secret, . . . ”), theparagraph can automatically be tagged by the highest-level tag. Taggingof the paragraph sets a range or a tear line, which can be selected fordifferent uses including a range for searching for key words for asearch operation.

Following is a process for creating automatic tagging with automatictear line tagger. (1) Select ranges of content (drop-down menu orpre-set range or manual highlight selection). (2) A range of contentmeans a contiguous content stream, which may be a phrase, line,paragraph, page, chapter etc. (3) Within the defined range, every dataelement and content element is monitored and categorized by a filter orfilters. The filters may be content filters like dirty word list ofothers. (4) Every data element is categorized contextually (namelocation, phone etc.). (5) Every element may be also labeled tagged andcategorized according to other categorizations like securityclassification. (6) After categorization of every element, a process mayselect a categorization labile to define the content range. (forexample, a paragraph may have 10 words 5 labeled are as Top Secret, 3labeled as Secret, and 2 words labeled as Confidential. (7) The rangemight be labeled, tagged as Top Secret. (8) Rules can be set so thatwhen a combination of a few words appears the range, the range group istagged at their level higher or lower. (9) Every tag's contextualvalidity maybe checked and verified by an inference engine. Bycategorizing and labeling every data element, the invention in essenceassigns different weights to the different granular content items.

A.13 CATEGORY TAGS AS PLACEHOLDERS IN DATABASES

The invention provides for a system and method for substituting tags asplaceholders for removed data thereby creating the info infrastructurefor unstructured data. For security or privacy reasons, granular dataelements are removed from a repository, database, document, or file,etc. Tags are inserted as placeholders to replace the data that wasremoved to other storage locations. Data mining searches locate the tagsinstead of data. The benefit of this operational system and processenables data mining that will not produce the removed secured data butit will let the user know that the data does exists and what type ofdata it is (name, location, social security number, etc). Data mining ispossible based on tags organized by categories, not whole information inunstructured data files. As an example, a document may be processed toshow: (a) Date birth xxxxxx; (b) Mothers name yyyyy; and (c) Schoolzzzz. The data categories (birth, mother's name, school) is listed, butthe granular data elements are in different locations—and the owner ofdata needs to trigger the release of different granular data elements.

Further, the tagging and sec-con and SC data extraction can be used topopulate a taxonomic database (or non-taxonomic database). The tags orlabels are the filed names in the database. The structured data in thedatabase will be the extracted data. Therefore, the sys-admin createsthe info infrastructure with the present process.

A.14 CONTROLLED RELEASE—BASIC

The controlled release of information (see reconstruction aspects insections B.7, B.8 and the applications in sections D.2-D.14) results inthe following unique features and characteristics: Layered build-up ofdata streams, layer upon layer; Granular data release from a third partyrepository; Creation of modular data components from granular datastreams; “Rolling” exposure of granular data streams; Controlled releaseof extracts after verification; Multi-purpose usage of filtering andextraction process; Filtering and copying “Selected data;” and, Acombined process of extracting part of the “selected data” and copyingparts of the “selected data.”

A.15 LAYERED BUILD-UP OF DATA STREAMS, LAYER UPON LAYER

The present invention provides a system and method for the controlledrelease of granular data streams which release builds up, layer bylayer, a reconstructed document or data stream. The controlled releaseof data or granular data streams—layer by layer—forms a new, whole orpartial document. A granular data stream is defined as extracted,granular data elements and/or the remainder data filtered from anoriginal data stream. The filter may process the original data stream bycontent filter or other filtering methods. Controlled release ofgranular data layers during reconstruction can be done in a document,data stream, and website, database or other format. Each layer is builtupon the other one to enable a view of a full or partial document. Theinvention provides a system and method for controlled release of dataand granular data streams after verification and validation before therelease of each layer.

One aspect of the present invention focuses on release from storage ofgranular data pieces. The security risk is reduced by storing smallerand more granular pieces. The attacker needs to access few stores topiece together all the content Layers of data of the original documentdata stream may be released at once or at different times. The granulardata stream is released into a file with existing content (remainder, orsanitized file) or creating a new file. See section D.6 for a sanitizeddocument reconstruction. For security reasons and to avoid covertchannels, Trojans and other attacks, the granular data layers can bereleased onto different screens in the same computer or differentscreens on different computers. See sections B.7 and B.8. The build-upcan be done in a way that resembles build-up of modular components.Different layers can be built up in different sequences.

A.16 GRANULAR DATA RELEASE FROM THIRD PARTY REPOSITORY

The present invention provides a system and method for granular datacontrol and release from a third party repository. Users control theirown private data-release from 3^(rd) party or government repositoriesand databases. When a user wants to release information, he or she willrelease it in a controlled release fashion. A higher level of securityand privacy is delivered in situations where the data is split into datastreams and stored in different memories and or locations. When theinformation is dispersed, the chances for leakage are minimal. The usermay release different data streams from the different storage locations.

To deliver a higher level of security the user may choose to route thedispersed data streams in different communication paths. For example:deliver critical information in parts over the phone line, cell line,email and fax.

A.17 CREATION OF MODULAR DATA COMPONENTS FROM GRANULAR DATA STREAMS

The system and method provides for the creation of modular datacomponents from granular data streams. Splitting original document ordata streams into granular data streams enables the user or sys-admin tobuild and maintain a component infrastructure for internal usage (withinthe enterprise) or for external usage (to others). A granular datastream is defined as the extract and/or remainder data filtered from anoriginal data stream. The filter may process the original data stream bycontent filter or other filtering methods.

The trend of implementing Service Oriented Architecture—SOA, in webapplications teaches the need for modular code, the benefits of codereuse and other added value. In the same vein, splitting originaldocuments or data streams into granular pieces and creating a modularframework brings different added value and advantages which include: (1)parts of the data stream can be used by the originator, other parts byother users or machines to construct other data streams; (2) thegranular data streams may be used as tags or metatags; and, (3) thegranular data pieces may be used as elements to feed search engines.Modular data components may be granular data streams or pieces, granulardata pieces with hooks, granular data pieces with tags etc. Hooks aremetadata that relate one data element to another data element or file ordata object. The modularity enables data pieces to be easily combinedinto a larger data structure where they may be reused.

A.18 “ROLLING” EXPOSURE OF GRANULAR DATA STREAMS

The invention provides a system and method for flexible content accessbased on “rolling” granular data exposure with decryption for betterworkflow. In situations where information must be accessed on an ongoingbasis in plain text like search operations, encryption may be a majorhindrance. The need to decrypt the encrypted plain text and then toencrypt it again, and do the same loop many times, is a major obstacleto work flow and performance. The invention introduces a solution basedon creation of (1) granular pieces of data (2) a distributed storageframework as a way to deal with the need to encrypt yet not overwhelmthe processing and other computing workflow. The system creates granulardata pieces out of the original document/data stream. This is donethrough a process of content analysis, selection, extraction anddispersal to distributed storage.

The granular pieces may be encrypted. The added value of the granularityeffect is that there is no need to decrypt the whole original documentat once pieces of granular data can be decrypted separately at thedifferent storage locations.

Operationally, the system may be pre-set to encrypt and/or granularlyextract and store based upon (i) event, e.g., save doc, log OFF, or (ii)time function; or (iii) user activated switch for encrypt and/orgranularly extract and store.

A.19 CONTROLLED RELEASE OF EXTRACTS AFTER VERIFICATION

The invention provides a system and method for controlled release ofextracts after verification and validation. See sections B.7 and B.8 andthe applications in sections D.2-D.14. One aspect of the inventionfocuses on release from storage of granular data pieces. The securityrisk is reduced by storing smaller and more granular pieces. Theattacker needs to access few stores to piece together all the content.It in noted that after their splitting from a document or data stream,granular pieces of data may be processed using the Information DispersalAlgorithm.

The invention provides a system and method for controlled release ofextracts after verification and validation. Controlled release ofextracts will take place at once or in stages only after verificationand validation of different identifiers including any of the followingor their combinations thereof: Identification code (PIN); Biometrics;Past history; Chain of title of data extracts; Past projects user wasinvolved in; Current projects user is involved with; and People user isinvolved with.

A.20 MULTI PURPOSE USAGE OF FILTERING AND EXTRACTION PROCESS

The present invention provides a system and method for multi purposeusage of the process of: filtering the elements in the data stream, theselected data is extracted, and dispersed to memories and reconstructingprocess—including an interface therefor. See, for example, section D.13and the web-based process. The process of document or data streamfiltering, extracting selected data, dispersing to memories andreconstructing process is created for different purposes. These purposesmay be presented to the user in a pull-down menu. The purposes include:(1) Archiving; (2) Compliance with policy and regulations; (3) Security;(4) Data searching/data mining; (5) Survivability; (6) Continuity ofoperations; (7) Replication/Copying; (8) Backup storage; and, (9)Information sharing. All potential applications or part of them can bepresented to users in various interfaces including a drop-down menu orin a different menu including a mouse menu. Presentation of a key oridentity card including credit card etc. may be requested to enableaccess to manual or automatic processing of selected choices. In eachcase, the original document or original input data stream data is beinganalyzed at the content level.

The process includes: (a) the original data or data stream is split intodifferent levels/pieces of data, created with the appropriate taggingand extracted to different memories. (b) split—the different data piecesare transferred to different memories, which will allow reconstructionor multi-level reconstruction.

A.21 FILTERING AND COPYING “SELECTED DATA”

The present invention provides a system and method for of filtering theelements in the data stream, resulting in selected data, creating copiesof the selected data and dispersing the copies to memories andreconstructing process—including a user selectable interface foractivating such operation. The process of document or data streamfiltering, extracting selected data, dispersing to memories andreconstructing process created for different purposes includes:

(a) in each case, the original document or original input data streamdata is being analyzed at the content level; (b) the selected pieces ofdata are copied then (c) appropriately tagged then (d) the differentpieces are extracted to different memory stores, which (e) will allowreconstruction or multi-level reconstruction.

A.22 COMBINED PROCESS OF EXTRACTING PART OF THE “SELECTED DATA” ANDCOPYING PARTS OF THE “SELECTED DATA”

The invention includes a system and method for multi-purpose usage ofthe process of: (a) filtering the data elements in the data stream, (b)resulting in selected data, (c) extracting and removing part of theselected data while (d) copying other parts of the selected data forfurther processing. Copying selected data, extracting data, dispersingto memories and reconstructing process—are all actuated by a usercontrolled interface—such as a pull down menu.

For each of the purposes specified below, the original document ororiginal input data stream data is being analyzed at the content level.The selected pieces of data are either: (1) Copied then appropriatelytagged then the different pieces are extracted to different memorystores, which will allow reconstruction or multi-level reconstruction.Or (2) Split from the original document the content pieces may beappropriately tagged then the different pieces are extracted todifferent memory stores, which will allow reconstruction or multi-levelreconstruction. Presentation of a key or identity card (including creditcard data) to enable automatic processing of selected choices.

A.23 DATA MINING SEARCH RESULTS BASIC—DISTRIBUTION AS MULTI-LEVELSECURITY VERSIONS

The present invention can be configured as a data mining tool for searchresults which are distributed in multi-level security versions, asearch-creating automatically key words, a continuous data mining withquery refinement; an automatic continuous search with divergence andconvergence, and with distributed data mining.

The present invention provides a system and method for search resultsdistributed as multi-level versions. The KE engine (section C.2-C.23)can be combined with the multi-level security system per sectionsD.4-D.9. There is a need to enable a user with a security clearancelevel and a specific role to search documents, data streams, data basesand data repositories that are at varying security levels which he isnot allowed to access. The problem can be overcome by allowing a searchto takes place in the different databases. This invention provides asystem and method for mult-level scrubbing and distribution ofmulti-level search results

Search takes place in all data bases—a multi-level security guard ispositioned at gateways scrubbing all results generated from thedifferent sources/databases. User gets results scrubbed to his level ofaccess. The system can be positioned as a guard at the gateway of thesearch process. The system scrubs the search results or scrubs a copy ofthe results in such a manner that every user gets a version of theresulting document according to his clearance level, or role. Theresults of the search are (1) sanitized and versions are created thatcorrespond to the user security level and role or (2) a sanitizedremainder document is prepared and granular data will be released intoit from storage store or stores, local or remote.

Presented herein is a multi-level filtering and dissemination system andmethod of search results and search hits. The searching takes place indifferent security level databases. In situations where the search isconducted by a search engine that delivers at first summaries or hits ofdocuments, the following process will take place: (A) The result/hitspages are opened. (B) Each page is filtered; (C) Selected words aretagged and extracted; and (D) MLS—multi level security is enforced fordistribution to the users. Each user sees only what his securityclearance level permits him to see. The summary of the search hits mayalso filtered creating words that are tagged by different securityclearances. Each user then will see words in the summary of hits that heis allowed to access at his clearance level

A.24 DATA MINING—A SEARCH CREATING AUTOMATICALLY KEY WORDS

The present invention provides a system and method for automated searchengine with automated creation of key words. Reference is made tosections C.2-C.23, particularly C.9, C.10, and C.22, C.23. Documents arethe core of the “knowledge economy.” Finding documents in anorganization or enterprise is a tough challenge because of the ocean ofinformation. The challenge is even harder trying to find the neededinformation outside the organization, including searching open sourceinformation on the Internet. Since it is hard to locate relevantinformation studies show that more than 50% of data existing in theorganizations is recreated. A filtering system is presented herein toremove and to locate relevant documents/data streams from dauntingvolume of documents in an organization as well as open source data.

The system and process can be pre-set to automatically trigger a keywordsearch in which the keyword/search items are not submitted by a personinto the search engine, but submitted by an automatic machine processafter a body of content, like a document/paragraph or other data stream,is fed into the machine. The machine analyzes the content of the datastream and, based on preset filtering processes, locates the key wordsand submits them to a search engine or multiple search engines.Different key words may be added to the search that are not included inthe processed data stream. The triggering starting point: A search maybe triggered by a user dragging a document/paragraph/line into thesearch engine, copying content into the search engine, or through anautomatic feed of data into the search engine, for example an automaticfeed of news articles. Selection of search key words: The systemanalyses every word character icon image and categorizes them. Thesystem selects the ones, which match the predetermined rule set androutes them to a search engine for a search. Words, characters and iconsimages that are not recognized by the filtering and categorizationprocess may be categorized as “unknowns” and sent as key words to thesearch engine. Reduction of relevant search results: Results of thesearch may be processed (by use of a filter or filters) to reduce noiseand deliver most relevant results.

A.25 DATA MINING—CONTINUOUS DATA MINING WITH QUERY REFINEMENT

The present invention provides a system and method for automatednon-stop data mining cycles with query refinement. Data mining triggeredby key word or document feeding into search engine. Search results arefiltered again to find new keywords so another search will take place.As results are submitted they are automatically filtered by presetfilters and new keywords are located and sent to search engine.Selection of key words: All key words may be sent to search engine. Itis more practical to implement an n word automatic (possibly manual)selection of most relevant keywords for the search process. Differentfilters may be used. Categorization filters may be used—if data inoriginating document (or other process related documents) wheregranularly categorized—selection of specific categories may be enforcedin the filter. For example: “select only names and locations”—This willreduce the number of key words only to those, which are names andlocations. Other filters (sieves) may be used for example; “select onlyitems with a high sensitivity level tagging” then only items such as TopSecret will be selected. Selection of range for key words: To focus oncreating key words for search from specific content area within the datastream ranges of operation will be set. The range defines an area withina document; data stream within that area key words for search will belocated, selected and fed to search engines. Ranges can be createdautomatically based on categorization of items within the data stream.

The inventive system and process may also be set for dispersal tostorage of a federated data search/mining operation. Filtering optionsand combinations for sending keywords to search engine include: (1) Wordlist—a dirty list filter; (2) Group word list—a list of groups of wordsrepresenting some context possibly a concept; (3) Categories—filterbased on selection of specific pre-selected categories (like names,locations); (4) Security level—a filter based on selecting a specificsecurity level (Top Secret tagged content or Secret tagged content); (5)Range—A meta filter determining what content area in a data stream wouldbe the focus of the search. (tear line); (6) Scope—A filter selectingwhat extracted words in the selected range; and, (7) Other key words fora joint search may be presented.

A.26 AUTOMATIC CONTINUOUS SEARCH—WITH DIVERGENCE AND CONVERGENCE

The present invention provides a system and method for storage with anautomatic data search/mining operation (convergence). Automatic datasearch and mining operations in a data store is triggered by dataresiding within the data store. Keywords are selected and sent to searchengine or engines. Data search mining is triggered by key word ordocument fed into search engine. Search results are filtered again tofind new keywords so another search will take place. Other inputstriggering an automatic search may be sent from other sources includingother storage stores, thereby creating a federated search. The datasearch mining operation may continue on an ongoing basis. The user hasthe ability to set the system ON for a continuous, non-stop cycle offiltering keywords and feeding them to search engines. The user may setthe system ON for a set number q of search cycles. (e.g., stop after 5cycles). The user has an ability to control cycles by establishing atime (cycles stop after 30 minutes). All results are filtered to reducenoise and achieve a high level of relevancy. Results may be matched towords in the originating data stream to insure relevancy. Data searchresults or other input of the storage store may be sold, barteredexchanged with others.

Currently, data storage is cheap yet there is minimal usage of data instorage. When data in storage is used, the data has added value for theenterprise. The system data mines unstructured data and enables theenterprise to sell search results, or combine newly discovered data withenterprise data, for a synergistic resultant data of higher value. Thesale or barter of data may be for a price.

A.27 DISTRIBUTED DATA MINING

The present invention provides a system and method for distributed datamining. Running search operations from dispersed locations both forsecurity as well as better coverage and expansion of select content SCdata. For security reasons, searches should take place from differentdispersed locations, in order not to enable a compromise of the data.(for example, a compromised insider at Google can know all the subjectsthat are searched by a person or organization).

A.28 CONTENT INFERENCE ENGINE—BASIC

The present invention can be configured as a content inference enginewith the following features: (a) an inference engine searching for theunknowns; (b) securing information sharing with an inference engine; (c)an automatic web data inference crawler robots; (d) changeable dataclassifications based on content inference threats; (e) searchoperations creating new continuously growing digital data streams; (f)distributed searches from different storage node; and, (g) storage ofseparated data pieces in one location

A.29AN INFERENCE ENGINE—SEARCHING FOR THE UNKNOWNS

The present invention provides a system and method for Data InferenceEngine with Divergence data search focusing on “Unknown” data, that is,data which has not been identified by the initial set of filters used toprocess the source doc or information file. The objective is to locatehidden data and to infer data therefrom that is identifiable andrelevant. The system locates documents, which they are related to theinitial document, each other by context or concept. The search takesplace in structured as well as unstructured documents. See the KE enginein sections C.2-C.13 and C.20-C.23. Data search mining is triggered bykey word or document feeding into search engine. Search results arefiltered again to find new keywords so another search will take place.Known key words are removed by the filters or given low priority—unknownkeywords are sent to the search engine.

The user can set the system ON for continuous, non-stop cycle offiltering keywords and feeding them to search engines, set number ofsearch cycles (e.g., stop after 5 cycles), and control cycles byestablishing a time period. All results are filtered to reduce noise andachieve a high level of relevancy. Results may be matched to words inthe originating data stream to insure relevancy.

A.30 SECURING INFORMATION SHARING WITH AN INFERENCE ENGINE

The invention provides a system and method to establish an informationsharing environment counter-balanced by inference searching. Informationsharing environments may be open to security risks therefore it must becounter-balanced by content inference searching and testing operationsthat will enable the sys-admin to identify a threat so that the securitythreat can be reduced. Therefore, search engines/inference engines mustwork constantly in a plain text environment. Different informationsharing projects have been shut down because the data acquisitionsystems where a threat to privacy. Data mining is a constant threat toprivacy and leads to many instances of identity theft. This inventionprovides a system and method for protecting privacy and security in datamining and information sharing operations.

The invention protects privacy and security by controlling the access tosensitive content. The sensitive information is defined by the inferenceengine. Documents and data streams are filtered by the inference engine,granular data is selected, (and may be extracted to distributed stores).Granular pieces of data are released by a controlled mechanism to avoidsecurity and privacy breaches.

A.31 AUTOMATIC WEB DATA INFERENCE CRAWLER ROBOTS

The present system is a method for automatic crawling robots—spiderslocating data and locating threats to data and minimizing the threatsincluding scrubbing web sites. Automatic agents, robots, mobile codeanalyze data in documents, data bases and data streams. Using inferencecapabilities, which may include artificial intelligence, the systemlocates users' private data which may lead to a threat of compromisinghis other data his privacy etc.

The system can be configured to remove the threatening data. Removal ofthreatening data with the option of reconstructing it back uponpresentation of proper identification and security clearance. Automaticrobots, mobile code, may be used to crawl web sites, data centers,directories, and data streams on a continuous or part time basis.Consumer reports may be generated and sent confidentially to users.Thereafter, the user may demand that the web sites, data centers,directories, and data streams maintain user data in a confidentialmanner consistent with data privacy laws.

A.32 CHANGEABLE DATA CLASSIFICATIONS BASED ON CONTENT INFERENCE THREATS

The present invention provides a system and method for flexiblechangeable classifications and security classifications, with optionalinference penetration testing. A system and method enable the user tochange security classifications on the fly. Security levels of adocument or data stream are upgraded or down-graded based on the resultsof inference tests. Due to the overwhelming ocean of data in anorganization and in opensource data, data classifications, andespecially security classification, should be changed on the fly as newdata is being integrated discovered and fed from search results. Whatwas “secret” in the past (and important and critical) may now be publicinformation and the value of the data may be lost. Data that wasclassified at a low priority may need to be upgraded as new informationis analyzed. The system can be automated to change classificationlevels. Classification levels change and tags or labels change as wellas security classifications based on results of content inferencingpenetration testing. This present invention provides a system and methodfor on the fly change of classification including securityclassifications: (A) Data that is classified at a specific securitylevel for example “Secret” may be used by an attacker an insider oroutsider to search that classified data against open source searchengines or other databases. The results of such an inference attack maylead to new information about the classified document that is moresecretive than the security level assigned to it. For example, aninference search might yield names and locations that were extractedfrom the “Secret” document in such a case the base classified documentshould be classified at a higher security level than “Secret.” Thesystem presented will conduct inference penetration test based on thelevel of sensitivity of the located information the security clearancelevel will be upgraded.

Since data is constantly changing, real time analysis must take place onan ongoing 24 hours effort. The user requesting access to the data willreceive data not based on the old security clearances of the documentbut based on the up to the minute update. The system and method may beautomated based upon enterprise informational policies.

The present invention provides a system and method for automatic datamining operations creating new intelligent digital datastreams/entities. Automatic data search mining operations create awealth of information. A wide net is cast and many documents and datastreams are located. A filtering process is used to deliver mostrelevant results. On-going search operations create a digital datagrowth which can be described as a growing tree with branches that areconstantly growing and branching out. In essence new intelligent digitaldata streams are created on the fly around a “content target.” The term“content target” is defined as a “concept” which may grow in differentdirections based on results of the data search operations. The inventionprovides a system and method for automatic addition of relevant searchresults to the “content target.” Automatic scanning of data within acomputer, data base, or network nodes results in analysis of content.Identification of, among other things: (1) what are the criticalelements of the data and/or; (2) what elements of the data need morerelevant data including updated data. Then, the system automaticallysearches for relevant data to update or to add to the targeted data. Theaddition of data can be in the form of an attachment, a URL, a meta tagor data in different formats or linked by different methodologies. Theanalysis process can be done in various ways including the use ofcontent filters.

A.34 DISTRIBUTED SEARCHES FROM DIFFERENT STORAGE NODES

The present invention provides a system and method for distributedsearches and establishes an interaction between distributed storagestores with data mining operations. Distributed storage stores eachindividually carrying data mining operations can yield a vast body ofdistributed data and knowledge. An infrastructure for interactionbetween stores is presented herein. The infrastructure enables sharingof data sharing of key words in order to enable better morecomprehensive data search operations. For example, if one store storesdata on a subject A and another store stores data on subject B, sharingof key words or data between the stores will enable better key wordsearches. In essence, searching in different stores (each one with itsown subject matter) results in more robust search results.

A.35 STORAGE OF SEPARATED DATA PIECES IN ONE LOCATION

The invention provides a system and method for storing different dataextracts in one storage location. The system and method stores extractsof a data stream in different memories within one storage location.There is a major difference between splitting a document or a datastream and placing its parts in one storage location and this invention,which deals with placing extracts of a document or a data stream in onestorage location. This invention deals in a situation that a whole dataasset was already parsed—and split into a “remainder” and “extracts”.What is transferred to one storage location is not all the pieces of awhole document or data assets but partial part of the whole the“extracts.” The storage location can be one computer one storage node orother medium that is located in one physical location. To overcomesecurity risks, the extracts in the different memories within the onestorage location may contain protective elements which make it very hardto match the different extracted pieces together to create an originaldata stream. Those protective elements may include adding a substantialamount of additional data assets that will create chaos. It will be amajor obstacle for an attacker to locate the original extracts and piecethem together. Another protective measure is to encrypt each of theextracts in the different memories and store their encryption keys in adifferent storage location. An attacker who will penetrate the extractstorage location will need the encryption keys in order to access thecontent. The encryption key or keys may be split in a secret splittingscheme. In essence, m out of n keys will be needed to access the contentof the extracts in the different memories. The storage location can bealso software storage without a physically defined location.

A.36 STORAGE OF GRANULAR DATA STREAMS—BASIC

The present system and method can provide for configuration of granulardata streams and dispersal to distributed storage stores; shuttling ofdata and granular data streams between distributed storage stores;replication of granular data streams with minimal security risk; and,anonymizing classified and unclassified storage stores.

A.37 CONFIGURATION OF GRANULAR DATA STREAMS DISPERSAL TO DISTRIBUTEDSTORAGE STORES

The present invention provides a system and method for control of dataextracts in dispersed storage environment. A granular data stream isdefined as the extract and/or remainder data filtered from an originaldata stream. The filter may process the original data stream by contentfilter or other filtering methods. Granular data streams are filteredand the results or parts thereof form bigger data streams. The resultinggranular data streams may be extracted or copied for further processingand storage. To achieve high levels of efficiency as well as security, aset of configurations is required to enable maximization of benefits inprocessing and storing granular data streams a distributed environment.Since granular data streams are part of a bigger data stream—thegranular data stream location needs to be entered into a map ordirectory of the full data stream to maintain context.Configurations-may include: how many storage locations are available,what would be transferred to each storage location, and when. Thestorage nodes may be simple storage nodes, or smart processing storagenodes. Enterprise policies set data processing rules for the infoinfrastructure based upon who, where, what, when, why, and how much-howlittle (quantity).

A.38 GRANULAR DATA STREAM TRANSPORT

The configuration of granular data streams transport to storage mayinclude a selection of what type of data streams will be sent to whichstorage. The selection may be done randomly, or according to the typesand or categories of content, specific key words in the granular datastrings are going to be shuttled. The data storage facility may beselected at random or via an algorithm to enhance security. Storage bydata class is discussed later. The configuration will also includelocations of storage on the network. Special configurations will need tohandle searches combined from key words in the different extracts, whichmight be in the same storage node or on other nodes. The configurationswill cover the different possible permutations of combinations of whichkey words in the extracts may be sent as a combination to searchoperations and in what order.

A.39 SHUTTLING DATA AND GRANULAR DATA STREAMS BETWEEN DISTRIBUTEDSTORAGE STORES

The present invention provides a system and method for data assetinteraction in a distributed network. Shuttling of data, includinggranular data streams, between distributed storage locations for datamining purposes (and other purposes such as add on to other datastreams) causes chaos which increases security. Creating networkpaths/“road infrastructure” between storage stores so data and granularstreams can move (1) when requested, or (2) on a scheduled basis, or (3)upon certain events (see attack application in section D.3).

The method and system also process data and granular data streams fromdifferent distributed stores. The combined processing of the data mayinclude cleaning the data, extracting, and packaging it for data miningsearch operations and other purposes. Artificial intelligence may beused in this process. Creating a “chamber” or “compartment” forprocessing data and granular data streams from different distributedstores is possible with the system. The combined processing of the datamay include cleaning the data, extracting, and packaging it for datamining search operations and other purposes. Artificial intelligence maybe employed.

A.40 REPLICATION OF GRANULAR DATA STREAMS WITH MINIMAL SECURITY RISK

The present invention provides a system and method for granularreplication enabling the user access to many copies with minimalsecurity risk. It is common practice that to achieve reliability thereis a need to store a few copies of the data. The problem is that thebigger the number of copies made the bigger is the security risk. Theinvention splits a data stream into granular pieces replicates thosepieces and disperses them to distributed storage. The security risk isminimized and can be controlling the size of the granular data piecesand their number.

The invention also proposes a new architecture for storage on theinternet. The invention enables a user to make as many copies as hewants of a document or data stream with minimal amount of security riskIf a storage node is attacked a small granular piece will not pose aserious threat. A small granular piece does not convey all the substanceof the original document/data stream. If the replicated piece is smallenough the attacker will find it useless because it is out of context.For example, a granular piece of data which is a name only can't createa serious threat because it is out of context. Other stores need to beattacked successfully to access their data to give context to the smallgranular data piece. The security risk of having many copies can bereduced by the user decreasing the size of the granular pieces anddispersing the different pieces to different distributed storage store.

A.41 ANONYMIZING CLASSIFIED AND UNCLASSIFIED STORAGE STORES

The invention also provides a system and method for not labeled,non-security classified security storage stores, in essence. incognitostorage stores. The storage stores may hold highly sensitive secret databut are not labeled with a security label or identifiable risky tag toavoid attacks. The classification of the storage store may be recordedat a distant location, which can be a registry, an index or a map. Dataor granular pieces of data (split from a data stream) are stored indispersed storage locations including Internet URLs. The differentstorage locations are not labeled or not labeled in a way that will leadto a possible attack. Sensitive storage stores that are commonly taggedwith security or hardened classification at their physical location,like Top Secret or Secret, will not be tagged, tagged in a way thatmaintains their anonymity. For example, the data is labeled or taggedwith a number, in a way that its content will be unknown to potentialattackers.

A.42 CONTROLLED RELEASE OF GRANULAR DATA STREAMS—GRANULAR DATAMANIPULATION AND CONTROL

The present invention provides a system and method for granular datacontrol and manipulation. Granular data control is defined as: theability to manage manipulate and control granular pieces of data withina data stream or information file in order to achieve certain results.The result of such a process is the creation of new granular datastreams or structured data. A granular data stream is defined as theextract data and/or the remainder data filtered from an original datastream. The filter may process the original data stream by contentfilter or other filtering methods. The resulting data or data stream cannow be used or manipulated for different purposes. The resulting datacan be presented in different formats to enable maximum datamanipulation. For example, in one format, the resulting filtered datacan be put into a matrix as structured data. Assuming a content filterwas employed, and the filter also included tagging or labeling forcategories (taxonomic filter), one axis of the data matrix may representsource doc markers and placement markers, the other axis may representcategory groups (names, locations, and social security numbers), and thedata in the matrix shows the critical sec-con or SC data, that is thegranular filtered data elements form the source doc.

The granular content elements in the matrix can be manipulated manuallyor automatically based on pre-set rules and or configurations. Granulardata manipulation and control enables the user to achieve differenttasks including: (a) Privacy; (b) Security; (c) Secured Informationsharing—Multi level security/cross domain data sharing; (d) Compliance;(e) Survivability; (f) Continuity; (g) Secured backup; (h) Granular datareplication; (i) High relevancy searches; and, (j) Creation of tags andmeta tags.

The process for creating the info infrastructure described above in thisPart A (A.1-A.3) establishes this granular data control and manipulationand use of the system tools to create the structured data and toimplement the enterprise policies for tasks a-j with pre-set dataprocessing rules.

A.43 SANITIZATION PROCESS WITH INFERENCE FILTERING—BASIC

The present method ad system can implement a sanitization processassociated with inference filtering of the source doc of informationfile. This process includes (a) process for sanitizing data on thenetwork for privacy security and survivability; (b) automatedsanitization of important data; (c) sanitization of non-important data;(d) release of granular data streams in lieu of access identifiers; (e)release of granular data streams in lieu of consideration; and, (f)secured release of granular data streams to multiple display screens.

A.44 PROCESS FOR SANITIZING DATA ON THE NETWORK FOR PRIVACY SECURITY ANDSURVIVABILITY

The present invention provides a system and method for sanitizing dataon the network for different purposes including: privacy security,compliance, information sharing and survivability. The sanitizationprocess includes the following process modules: (1) First stage:filtering the input, original data stream, and locating importantcontent items (sec-con or SC data) that needs to be masked or extracted.(2) Second stage: implementing an inference process to verify ifanything can be inferenced from the sanitized data stream. (3) Optionalthird stage: masking or extracting more data items from the sanitizeddata stream based on the threat results (inference analysis) of thecontent inference process. The risk analysis can employ the risk modulesfrom U.S. Patent application Ser. No. 10/396,088 filed Mar. 25, 2003which is incorporated herein by reference thereto. Alternatively, thefirst level output report in FIG. 12 c and the tree or second leveloutput report in FIG. 12 d for the life cycle engine may be employed toassess risk by inference. See sections C.9-C.13 and C.23.

The user's network and data storage facilities need to be sanitized todefend against different types of attacks. Granular data items areextracted from the network and stored at dispersed storage locations.See sections B.1-B.6; see also, sections D.3-D.6 and D.11. The objectiveis to make the network “formless” to achieve, from order, a creation ofdisorder, so the attacker does not know what and where to attack.“Formlessness” is the answer against virus attacks, Trojansworms/inference attacks, covert channels and insider attacks. The theoryis to erase the user's digital foot prints while finding and inferencingthe competitor's footprints.

The extracted granular data streams may be dispersed to storage or todistributed storage stores. The dispersed granular data streams will betapped or retrieved for (1) reconstruction of the document (full orpartial reconstruction); and, (2) to allow more data search needs.

A.45 AUTOMATED SANITIZATION OF IMPORTANT DATA

The present invention presents a method and system for automatedsanitization of important content (sec-con or SC data). Data, contentincluding data, and content over the network must be sanitized onongoing basis to stop insider attacks, Trojan and spyware, contentinference attacks and other damage. Minimal information is exposed withthe present system and methods. Optionally, upon presentation ofidentification by users, layers of data will be reconstructed. Forhigher security, information search operations need to take place indistributed storage stores/nodes, and users work mostly offline.

A.46 SANITIZATION OF NON-IMPORTANT DATA

The invention provides a system and method for automatic removal ofnon-relevant and unneeded content from information files. Automaticscanning and analysis of data within a computer, database, or networknodes is processed. Identifying data that might be not be relevant orneeded is located. Extracting that data and removing it to a storagelocation or assignment of destruction—deletion codes is part of the infoinfrastructure. Data that is not relevant or not needed may include: (a)out-dated data; (b) data not relevant to the core document or datastream; (c) data not relevant to the user's organizations task andmission; (d) data files having mistakes therein; (e) data that takes toomuch space; and (f) data that may confuse the users or others. The dataremoved to storage (extracted and remainder data) may be indexed forreconstruction at a later date.

A.47 RELEASE OF GRANULAR DATA STREAMS IN WITH INCREASING LEVELS OFACCESS IDENTIFIERS

The present invention provides a system and method for controlledrelease of layers of data for different access identifiers. A granulardata stream is defined as extracted data and/or the remainder datafiltered from an original data stream. The filter may process theoriginal data stream by content filter or other filtering methods.Controlled release of layers of granular data streams filtered from anoriginal data stream is permitted only after different accessidentifiers are presented. See sections B.7-B.8 and Part A herein above.As an example: release first granular layer like a paragraph afterpresentation of fingerprint, release of second paragraph afterpresentation of pin number, third paragraph released for retina scan,fourth paragraph released by secret sharing scheme need—other person tobe online with his identification to enable a release.

A.48 RELEASE OF GRANULAR DATA STREAMS AFTER OBTAINING CONSIDERATION

The present invention provides a system and method for controlledrelease of layers of granular data for different considerations receivedby the disclosing party or enterprise. A granular data stream is definedas the extract data and/or the remainder data filtered from an originaldata stream. The filter may process the original data stream by contentfilter or other filtering methods. Controlled release of granular datastreams including data pieces and data extracts is provided layer bylayer, after verification different criteria including: identity, pasthistory, verification of chain of title of the granular data stream,past projects user was involved, current projects etc. Release layer bylayer for consideration obtained by disclosing party. Forexample—release first paragraph if user inputs name and social securitynumber. Release second paragraph if requester inputs phone number.Release third paragraph if user inputs name of friends or delivers whathe is being asked to deliver. This invention provides also a system andmethod for controlled release of layers of data or granular data streamsfor different monetary considerations. See sections D.5 and D.6.Controlled release of layers of data for different monetaryconsiderations. For example: Release of first granular layer like aparagraph after payment of $1.50, release of second paragraph afterpayment of $2.00, third paragraph released for payment of $25.00, andfourth paragraph released after payment of a negotiated price. Thisinvention also provides a system and method Controlled release ofgranular data layers for a combination of different considerations.Controlled release of granular data layers for a combination ofdifferent considerations. Each granular layer may be released for adifferent type of consideration. For example: Release first paragraphfor a payment, release second paragraph after biometrics identification,release third paragraph after—filling some information.

A.49 SECURED RELEASE OF GRANULAR DATA STREAMS INTO MULTIPLE SCREENS

The present invention provides a system and method release of granulardata layers different computer screens. A granular data stream isdefined as the extract data and/or remainder data filtered from anoriginal data stream. The filter may process the original data stream bycontent filter or other filtering methods. To avoid Trojans, spy ware,and other attacks, granular data layers of original data stream arereleased into different screens in one or in separate computers. Forexample: Top Secret data layers will appear on a screen of one computerwhile rest of data layers (sanitized document) will appear on anothercomputer screen. See sections B.7-B.8. This invention also presents anoverlay of granular data streams stored in different memories whichmemories can be in one physical machine or archived to a higher level ofsecurity in different machines.

A.50 ECOSYSTEM BASED ON “GRANULAR DATA CONTROL”—BASIC

The present method and system can be used to establish an ecosystem withthe following features: (a) an ecosystem for information superiority andsurvivability—based on “granular data control;” (b) a system and methodfor minimizing data exposure; and (c) creating “digital doors” in a datastream limiting access the content of the stream or information file.

A.51 ECO SYSTEM FOR INFORMATION SUPERIORITY AND SURVIVABILITY-BASED ON“GRANULAR DATA CONTROL”

The present invention provides a system and method for an ecosystem fordelivering information for superiority purposes and for datasurvivability purposes, based on “granular data control.” In the digitalpaperless environment, the interconnectivity of networks and theInternet create new threats that can be compared to a battlefield. Toachieve information superiority and data structure survivability, thepresent system integrates the following concepts into one ecosystem. Thenetwork or ecosystem must be able to protect itself and the data itholds. It must conduct its own surveillance and generate information inreal time on threats, in essence, a command and control capability. Thesystem needs to be able to defend itself and sustain itself throughoutdifferent types of attacks and disasters. The core, basic concept forinformation superiority and survivability is the need for constant dataupdating, gathering and its secured sharing. To maintain competitivenessand defenses, ongoing and continuous information searching, data miningand data inferencing and information sharing operations need to takeplace throughout the organization as well as outside the organization. Acombination of some or all of the following processes and methods is thebasis for information superiority and organizational survivability: (A)Creation of granular data streams. A granular data stream is defined asthe extract data and/or the remainder data filtered from an originaldata stream. The filter may process the original data stream by contentfilter or other filtering methods. (B) Creation of content“Formlessness” by dispersing the granular data streams. (C) Dispersaland hiding of the granular data streams. (D) Compartmentalization of thedifferent granular data streams. (E) Access check points at the storagestores/nodes. (F) Reconstruction of granular data streams into theoriginal data stream or parts of it. (G) Multi level reconstruction ofdata. (H) Replication for continuity and survivability. (I) Constantsearch—for security and info superiority. (J) Content inference forpenetration testing as well as locating relevant information. (K)Secured information sharing. (L) Automatic continuous growth of a datastream by addition of new search resulted data to it.

To achieve information superiority an organization's data/content assetsto constantly grow and be updated in real time. Organizations andindividual data resources should be updated in real time. Acomprehensive search and data acquisition and organization is key to theorganization—like food and oxygen is key to humans and animals. There isa need for on-going search throughout the organization. Every node onthe in-enterprise's network must be engaged in searching data—in essencesupplying oxygen to the existing data assets. To deliver “competitivesurvivability,” the search capability needs to constantly locaterelevant information and update the existing data repositories.Presented herein is a system for continuous or part-time search all overthe organization assets and outside data resources including theInternet, all with updates in real time. The system and method is anecosystem for managing constant searches for (1) updating its datarepositories in real time, (2) creation of competitive knowledge, and(3) to secure itself from content inference attacks and other attacksand disasters. The interconnectivity of networks and the Internet createa digital battlefield. The answer against virus attacks, Trojansworms/inference attacks, covert channels employ the present process andmethod. The present system breaks away from the model of perimetersecurity and adopts a model that has been used in the battlefield formany years. The invention delivers capabilities to transform orderwithin the data content into disorder making it very hard for an enemyto attack.

The idea of “formlessness” is one of the core strategies of thebattlefield as expressed by Sun Tzu. Formlessness calls for minimizationof data, in essence, controlled release of data especially smallgranular pieces of data. In a formlessness environment, all extracts aredispersed all over in distributed nodes. The Ecosystem is in constantmode of searching to secure itself—against inference attacks—and to gainknowledge. Results of search are needed for security for (1) triggeringmore extractions, (2) more dispersal of data, and (3) gather moremisinformation which confuses the enemy—within or without theenterprise.

A.52 SYSTEM AND METHOD FOR MINIMIZING DATA EXPOSURE

It is impossible to project and foresee all possible threats to adigital environment. It is impossible to do all permutations of possibleinference attacks. Therefore, the invention presents a solution with theminimization of data. The invention provides a system and method forflexible content access based on rolling granular data exposure withdecryption for added security. Granular pieces of the originaldocument/data stream are dispersed to distributed storage nodes toenable a controlled secured environment for release of data. Thegranular data layers can be exposed one at a time decrypted instead ofexposure of a total document.

A.53 CREATING “DIGITAL DOORS” FOR ACCESS AND CONTROLLED RELEASE IN ADATA STREAM

The present invention presents a “digital door” system and method forcreation of doors in a data stream to achieve data control andmanagement via management of granular data. A system and method forcontrolling data and content within a computer, a network, server or website by management and control of data and data flows at the granulardata level. Content and data are monitored and controlled as they exitthrough access control points (see FIG. 1 a and sections A.1-A.3) aswell as when data enters the ecosystem, that is entry into the network,computer, servers, storage or web site. Content and data are monitoredby the system as the data is held in storage. The system may remove datafrom storage through the access control points for security reasons orfor other workflow and processing needs. Access controls or passwordsand sec clearance protocols may be applied sequentially, of concurrentlyof subsequently with respect to the extraction of data and storage ofdata. Hierarchical access controls are oftentimes used. Controllingmovement and storage of data by monitoring content, analyzing itsgranular parts when needed and masking or extracting granular datastreams and when needed and tagging data for infrastructure purposes.Enforcing information rights management (IR Mgt) processes to controlmovement and storage of the granular data streams is a function of thepresent system.

The system and method provides for management of data in a digitalpaperless interconnected environment. Controlling data is achieved byfocusing on the granular elements of data (character, word object,sentence paragraph etc.). When extraction is used, the system adds to“granular data control” the attributes of physical and logicalseparation of data. The security aspect may be further enhanced bytransfer of the extracted granular data streams into distributed storagestores. Creation of access control points or monitors in a data streammeans, among other things, extraction of data from a data stream withthe intention to reconnect the data extracted in whole or in part.Extracted data can be put in a package storage with or withoutencryption. The package may be tagged so to allow better management (IRMgt) of the extracted granular data streams enabling fastreconstruction.

The process may include: First stage: filtering of the inputted originaldata stream, locating important content items that need to be masked orextracted. Second stage: implementing an inference process to verify ifanything can be inferenced from the sanitized data stream. Third stage:masking or extracting more data items from the sanitized data streambased on the threat results of the content inference process.

A.54 GENERAL ASPECTS—BASIC

In general, the present process and system has the following additionalfeatures and characteristics: (a) automatic creation of a user orobject's identification profile; (b) copying of filtered granular dataand dispersing; and (c) combining extraction and copying for filteringof filtered granular data.

A.55 AUTOMATIC CREATION OF A USER OR OBJECT'S IDENTIFICATION PROFILE

The invention provides a system and method for automatic creation of auser or data object's identification profile. The profile will enableautomatic interactions that will locate relevant content for the profileincluding its update without a need for additional input. The profilemay be used as a guard and as a filter to enable or disable varioustransactions or communications related to the profile. The automaticbuilding of a profile is done in different ways including scanning ofdata assets within a computer, database, or network nodes. Analyzing thecontent including use of content filters and selecting patterns whichdefine the profiled subject. The system analyses each data asset,categorized its content, and give weights to the data asset. See the KEengine C.2-C.23. The data profile includes the type of content, whichparties interacted the data assets, at which time and location (audit orlog data). Who sent what type of data to the data asset, for whatreason? What is the security classification level of the data asset, howmany people have accessed the data, was it a private document or aprivate one etc.

A.56 COPYING FILTERED GRANULAR DATA AND DISPERSING

The system and method includes filtering data and copying the granulardata results and dispersing them to distributed storage. The processfilters documents and data streams and cops resulting granular dataitems for dispersal to distributed storage locations. Dispersal may bedone for different reasons for security, privacy, compliance, backup,continuity, survivability, backup or just close access or plainarchiving. Dispersal of copied granular elements can be done todifferent locations applications search engines. Any resulting filteredgranular data may be filtered again producing different levels ordifferent sub groups. The granular results can be categorized, taggedand sent to different storage locations for possible retrieval, fullreconstruction or partial reconstruction. The filtering process mayresult in creation of sub groups of granular items, in essence more thanone group of granular items. Those groups may be dispersed todistributed storage.

A.57 COMBINING EXTRACTION AND COPYING FOR FILTERING OF FILTERED GRANULARDATA

The system and method includes features for filtering data, resulting inparts of the granular data being extracted and the other parts beingcopied based on security threats, compliance requirements andoperational needs. Dispersal may be done for different reasons forsecurity, privacy, compliance, backup, continuity, survivability, backupor just close access or plain archiving. Dispersal of copied granularelements and extracted granular elements may be done to differentlocations applications search engines. Any resulting filtered granulardata may be filtered again producing different levels or different subgroups. The granular results can categorized, tagged and sent todifferent storage locations for possible retrieval, full reconstructionor partial reconstruction. The filtering process may result in creationof sub groups of granular items, in essence more than one group ofgranular items. Those groups may be dispersed to distributed storage.

B.1 BASIC OPERATIONAL THEORY OF SECURE OR SELECT DATA STORAGE

FIG. 4 diagrammatically illustrates the basic processes for establishinga secure storage of information, generally identified herein as “data.”The secure storage includes an extraction module and a secure storagemodule. “Data,” as used herein, includes any data object, e.g., text,images, icons, data element, moving images, multiple images, datarepresenting sound, video, electronic streams of information, etc. Soundbites and video images may also be extracted data. A source document 100containing data, sometimes referred to as a “plaintext,” is passedthrough a filter 102. Throughout this specification “source document”also refers to an information file containing data, particularlyunstructured data. The module described herein operates on securitysensitive content and operates on “select content,” wherein selectcontent is information which is deemed to be important to an enterprise.Select content may also be security sensitive content. In other words,select content has value, such as intellectual property IP value.Security sensitive content is deemed to be important in that itsunauthorized release is somehow detrimental to the enterprise. Althoughit is convenient to discuss and understand the invention herein inconnection with a plaintext document, the document 100 is a data object.It is not limited to an electronic document representing words. Thedocument 100 represents a data object that may be, e.g., text, images,icons, moving images, multiple images, data representing sound, videoetc. The term “data object” as used in the claims is broadly defined asany item that can be represented in an electronic format such that theelectronic format can be manipulated by a computer as described herein.The data object, or as discussed herein, the “plaintext” is sent to afilter. Filter 102, in a most basic sense, separates out common text orremainder data 104 from uncommon text, words, characters, icons or dataobjects. The security sensitive (or select content) words, characters,icons or data objects are separated from remainder or common text 104 asextracted text 106. It should be noted that although the word “text” isutilized with respect to remainder text 104 and extracted text 106, thetext is a data object and includes words, phrases, paragraphs, singlecharacters, portions of words, characters, whole or partial images,icons or data objects. In a basic implementation, filter 102 may utilizea dictionary such that words present in the dictionary (common words)are separated from the source plaintext document 100 and placed intoremainder document or common data file 104. The uncommon words(extracted-security sensitive (or select content) words), not found inthe dictionary, would be placed in an extracted text or extracted datafile 106. For example, a business may wish to impose a security systemon a contract document such that the names of the contracting parties(not found in the dictionary) and the street names (not found in thedictionary) would be stored in extracted data text file 106. The commontext or remainder data would be stored in remainder data file 104. Inthe illustrated embodiment, remainder data file 104 also includes placeholders which enables the extracted data to be easily inserted or setback into the remainder data file.

B.2 GENERAL OPERATION

FIG. 9 diagrammatically illustrates the major components of a flowchartfor the data security program and shows an extraction module, a contentfilter module and a secure storage module which can be modified toprovide a select content SC module. Rather than operating a securitysystem, the administrator (CIO) may operate a select content SC systemto gather important or potentially important content in one location,with an audit trial to locate the source of the select content or “SC.”It should be noted that this flowchart may be truncated to limit userselection of certain items. The system would be pre-set to contain thesefeatures. Step 230 initializes the system. Step 232 enables the user todesignate various levels of security for the activity which he or shewill soon engage. The system, in step 234, enables the user to definethe levels of security parameters. The following Security Table givessome examples of the type of security that may be available to the user.

Security Table to whom to where when (time of day, day of week, month,floating but predetermined time frame) why (purpose, match purpose toother security parameters or to certain predetermined criteria) how(through what medium (LAN, WAN, Internet, direct dial link), download towhat site or destination) how long (duration) the reconstruction processwill be permitted per each security clearance level how much (differentsecurity levels enable reconstitution of documents and data withdifferent amounts of secure data therein) timing systems may requiresynchronization for a standard clock (i.e., atomic clock)

As an example of a truncated or pre-set program, a client-server systemover the Internet may have URLs designating storage sites and an ASP 152(FIG. 6 ) controlling storage. In this pre-set system, the user does notselect the sites. The sites may be randomly selected by ASP 152. The ASPmay use artificial intelligence AI to locate secure extract data storagesites. AI or inference machines can ascertain (a) traffic oncommunications channels, (b) storage limit issues, (c) transmissionfailures in the communications links, and (d) the degree of securitynecessitated by exterior events, i.e., terrorism alerts, virus alerts,war, data security warnings posted by trusted sources, MicroSoft,Norton, NASA, DoD, CDC, FBI, etc. Higher security alerts trigger the AIconfigured storage locator and facilitator to locate memory stores inhigher secured places. These higher security facilities may be morecostly, may be located in more stable countries or on more stableservers and may have greater degrees of encryption capabilities.

The user, in step 326 can designate the location of the filter, thecommon storage area for the remainder data, the extraction data storageand potentially multiple data storage areas or segments. The user mayenable an AI filter design. Step 238 permits the user to engage ordisengage encryption and, if engaged, establish the degree of encryptionfor the system. Step 240 enables the user to define the parameters ofthe filter. The user can retrieve a preexisting filter or may define anew filter for each data security session. These filters may consist ofdictionaries or any type of compilation of words, characters, icon, dataobjects or pixel formation or any indication that can be perceived bythe computer system. These are called content filters. Granularextraction of data elements (or SC) in a data object may be permitted.Step 242 recognizes that the user either inputs a preexisting plaintextdocument or types data into the system. In any event, the plaintextdocument is fed through the filter. Step 246 extracts the security dataor SC data from the input document. Step 248 stores the extracted data.The extracted data may be encrypted prior to storage. Step 250 conductsan error check on the extracted data. This error check is helpful indiscerning problems in the storage of the data prior to closing down thedata security system. Step 252 stores the common data or the remainderdata. Step 254 conducts an error check on the common or remainder data.The decision step 256 determines whether the user has selected a“destroy filter” command. If not, the filter is stored with or withoutencryption in step 257. If YES, the filter is destroyed with a deletionroutine. Typically, deletion is complete erasure of all traces of thefile including, in high security systems multiple write-overs or discreformatting. Step 258 stores a map. The map may be stored locally orremotely as described earlier. This is the map module. The system endsin step 260. All traces of these data elements or objects may be swipedclean or removed from whatever computer system generated the dataobjects or processed them, other than the memory storage locations.Deletion of data also includes the concept of deletion of datatransmission paths, URLs, storage site locations and all temporarymemory stores. Deletion of file location in the root directory of harddrive 168 of computer 140 is preferable in high security systems.

FIG. 10 diagrammatically illustrates basic flowchart features for thereconstruction process. The reconstruction process, although not shownin the flow chart for the info infrastructure, is a module that runs inconjunction with the access-permission-control module. Step 302 acceptsa request to reconstruct the secured data. In a select content system, asearch for the SC is initiated. Step 304 queries a local map and thesecurity system or protocol. In a preferred embodiment the user wouldhave to input several passwords, one of them being a local password oncomputer 140. A local map which may be accessed only through thepassword, may simply identify the URL of server 152. Decision step 306determines whether the local password is acceptable. If not, and errorstep is indicated in step 307, the attempt to log on to the securitysystem is noted in step 309 (an audit trail), and the system eitherbranches to repeat step 311 or bars the user from further activity instep 313.

Returning to decision step 306, if the password is locally acceptable,the YES branch is taken and the system executes step 308 which releasesa reconstruction request to the common storage facility I-com 154 orA-corn 108 (FIGS. 6 and 4 and 5 ). The system in step 310 logs the userin, as well as time and date and the data regarding the request. In step312, a download from the common data storage is provided to RAM 166 orhard drive 168.

In step 314, a query is made to obtain the remote map from the remotesecurity system. The decision step 316 indicates that the user againsuccessfully inputs his or her security code. If not, error routine 317is activated, the password failure is noted in step 319 (an audittrial), and the user is given an opportunity to repeat in step 321 or isbarred or prohibited from further activity in step 323. In an SC mode,the requester may be required to pay money for the SC data. In the SCmode, a password may or may not be required. If the user has correctlyinput the security code, the system in step 318 releases the keys (todecrypt) and the map and releases the reconstruction request to theremote storage for the extracted data. This could be computer storageI-ext 156 or computer storage B-ext 110. In step 320, the user's accessto the extracted data is logged in along with the time and day and typeof data request (audit trail module). In step 322, the system downloadsthe extracted data into RAM 166 and/or hard drive 168 of computer 140.In step 324, an error routine is operated on the extracted data in orderto insure that the extracted data properly matches the common orremainder previously stored. Decision step 326 determines whether theerror routine properly generates the correct count or output. If not,the system in step 327 indicates an error, in step 329 the systemdeletes the common files and the extracted files and the system in step331 logs in the failed attempt. If the error checking routine on theextracted data is acceptable, the YES branch is taken from decision step326 and the system, in step 328, proceeds to display the plaintextdocument or to integrate the plaintext document pursuant to the securityclearance initially input by the user. Step 330 ends this process. Theend process may entail encrypting the data again and swiping clean alltraces of data objects from the memory stores and computer handlingunits. Of course, every use of encryption requires decryption of thedata prior to reconstruction.

The system may incorporate various types of security systems orroutines.

TABLE Types of Security Systems pass word pass phrase multiple choicequestions and answers initial, intermediate and subsequent securityclearance routines biometric security routines (voice, fingerprint,signature, eye or retina scan)

The reconstruction routines may be interrupted or the security systemmodules automatically activated or initiated upon the occurrence ofexternally generated triggers or upon certain predetermined conditionsor conditional events. See sections D.12 and D.14, for example. Limitedextraction, security clearance, release of data and reconstructionlimits may be imposed. Artificial intelligence (AI) engines, inferenceengines or neural networks may be implemented to vary the permittedlevel of reconstruction via the security clearances. In other words, theAI system, as applied to reconstruction, may, relatively independent ofthe filter and storage processes, increase the necessary security levelspermitted to access and generate full or partial plaintext recreation.

The display systems 220, 222 in FIG. 7 include CRT monitors, LCDscreens, projection screens and combinations of those systems.

The audit trail to monitor reconstruct and reconstruction attempts mayinclude adding a time/date stamp to the remainder data and/or theextracted data prior to storage and a cross-check to the audit trail logduring the reconstruction process.

Placeholders in the remainder document may be:

Placeholder Table blank spaces data symbols or elements “---“ or “xxx”false data clearly erroneous data “ABC Company” or “Baker” chaff or hashmarks messages bar code serialization data alerts links to other dataobjects null set indicators “[ ]” URL or website addresses

It is believed that the present invention is faster, duringreconstruction, than standard encryption techniques, on the order of 100to 1,000 times faster.

B.3 SYSTEM CONFIGURATIONS

FIG. 6 diagrammatically illustrates a personal computer or PC computersystem 140, a second PC or computer 142, and a third PC-3. PCs 140, 142and PC-3 are connected together via a network 145 (LAN or WAN) and arealso connected to an input/output device 146 that may be generallydescribed as a router or a server to an outside communications system.The input/output device 146 is connected to a telecommunications system148 which leads to Internet 150. The Internet is a global computernetwork. Internet 150 is coupled to a plurality of servers, one of whichis server 152. Server 152 may be designated as an application serviceprocessor ASP. Internet 150 also includes various computer memorystorage devices such as computer storage I-com 154, computer storageI-ext 156 and computer storage map 158. Computer storage enabling thestore of extracted data includes a security level clearance module 157.Similarly, map computer storage 158 includes security level clearancemodule 159.

As stated earlier, the present data security system can be implementedon a single personal computer 140. In this case, different memorysegments or hard drive 168 may be used for A-com and B-ext. Typically,PCs include a keyboard or data input device 161, a display 163, acentral processing unit CPU 165, a video board 167 having video boardmemory 169, a fixed disc hard drive 168, a RAM 166, and input/outputdevice 164, a removable memory media drive 162 a (may be floppy disc)and a removable compact disk (CD) read-write (CD-RW) device or drive160. Also, storage in EMP or electromagnetic pulse hardened data storesis possible. Print stores rather than digital stores provides anotheroptional layer of security. The system may include other removable diskdrives, tape drives, or flash memory units. Internal units CPU 165,video board 167, hard drive 168, RAM 166 input/output device 164,removable media (optionally floppy) drive 162 a and CD-ROM device 160are all coupled together via an internal bus 171. Bus 171 represents aplurality of buses as is known to persons of ordinary skill in the art.

One methodology of implementing the present invention utilizes distinctmemory segments which may be designated in one or more of the following:hard drive 168, memory in a removable disk in drive 162 a, memory in aremovable CD disc in CD-RW device 160, and, to a very limited extend,RAM 166. In this manner, the user may select, generally at the outset ofthe process, that the extracted data memory storage B-ext 110 be storedon a floppy (removable memory) via drive 162 a or a CD via CD-RW drive160. The user can then simply remove the floppy or the CD and carry itwith him or her. To reconstruct the data, the operative program,generally discussed above would have access to the floppy or the CD andparticularly the memory location of the data on the floppy and the CD inorder to reconstruct the entire plaintext document 100 (see FIG. 4 ).Alternatively, different portions of hard drive 168 may store A-com andB-ext. Of course, the computer system may utilize tape drives andmemories or flash card, programmable memory.

In a local area network or wide area network implementation, PC 142includes memory similar to memory units described in PC 140 and a memorysegment may be set aside in PC 142 separate from the common data orremainder data storage typically placed on hard drive 168 in PC 140. Asa further expansion of the present invention, the extracted data (thatis, the high security data), may be stored on computer storage I-extmemory unit 156 via Internet 150, telecommunications system 148 androuter/server 146. In this manner, the common data or remainder data isstored on hard drive 168 and the highly sensitive data is stored offsite in a secured location. Access to that secured location may belimited via security layer 157. If the user implements an encryptionsystem (see encryption e 118 in FIG. 4 ), the extracted data is furthersecured by the encryption during the transfer from computer 140 throughnetwork 145, router/server 146, telecommunication system 148, Internet150 and ultimately to computer storage I-ext 156.

The present invention may also be embodied utilizing an ApplicationService Provider on server 152 and in a client-server network.

An implementation of the present invention over Internet 150 most likelyincludes the use of a uniform research locator or URL for map memorycomputer 158, computer storage I-ext 156, computer storage I-com 158 andASP server 152. In a client-server environment, server 152 acts as aserver generally commanding the operation of client computer 140. Ofcourse, persons of ordinary skill in the art recognize that the servermay be located on the local area network 145 rather than beinginterconnected with Internet 150 as shown in FIG. 6 . The claimsappended hereto are meant to cover the alternative embodiments.

As an example of a client-server or web-based implementation of thepresent invention, the user at computer 140 may define the filter 102 asdescribed above, and input data (plaintext) via keyboard 161 or loadplaintext data from drive 162 a or CD-ROM drive 160 into RAM 166. In anyevent, whether the plaintext data is input via keyboard 161 or copied oraccessed from removable media drive 162 a or CD-RW drive 160, theplaintext data is filtered as discussed above in connection with FIG. 4. Prior to filtering, it would be appropriate for the user at computer140 to identify where the remainder data or common data will be storedand where the extracted or high security data would be stored. A simpleprogram may automatically select the secure store location. The systemis sufficiently flexible to enable the user to select local storage ondifferent memory segments of PC 140 (hard drive 168, removable(optionally floppy) drive 162 a, CD-RW drive 160) or be flexible enoughto enable user at computer 140 to designate off site storage of the highsecurity data (extracted data) and/or the common or remainder data. Anautomatic store routine may only require the user to accept or reject topreferred first security level, second security level and highersecurity level stores. The off site data storage process may includeactivating server 152 and enabling the server to take over the processdirectly from user 140. In other words, the user at computer 140 couldcall up the URL of the server 152, the server could request certain userinformation (user name, password), and would request data from theclient computer to establish the filter pursuant to input selected bythe user. The client computer may (a) filter the plaintext thereat or(b) send the data to the server for filtering. The server could storedata either locally on computer 140 or remotely at computer memories154, 156. After storage of the data at any of these locations, theserver 152 may establish a map and store the map in memory location 158.Of course, remainder data (cleansed, plaint-text data) and the map maybe stored at ASP 152 or client computer 140. The map, if stored at mapstorage 158, may be downloaded to the user at computer 140. The filtermay be stored at computer 140 or may be stored at a secured location onserver 152. Alternatively, the map could be destroyed on user computer140. The filter could also be destroyed on user computer 140. Of course,the filter could be stored in a fourth remote location (not shown),different from I-com 154, I-ext 156 and map computer memory 158. Storageof the map and decryption keys is a critical, high security task.Appropriate security measures should be utilized to protect those items.Local removable memory storage on disc in removable drive 162 a or discin CD-RW 160 may be reasonable. All traces of the map, the filter, theencryption key, the extracted data, and possibly the remainder data maybe scrubbed or deleted from all computer memories (by write-over or discreformat routines) other than the “com” and “ext” storage sites.Deletion of all URLs, links, x-pointers, etc. is also recommended forhigh security applications. Deletion systems are known to persons ofordinary skill in the art. For multiple security levels, multiple website for storage of cleansed plaintext, first, second, third and highersecurity level extract text is preferable. Where the community ofinterest has access to the targeted and protected data via the Internet,multiple secured storage locations, multiple stores for filters, forencryption keys and for maps locating the secured stores is provided bymultiple storage locations distributed throughout the Internet.

To reconstruct the document, the user at computer 140 would be requiredto call up the URL of server 152 and input the appropriate securitycode. The server 152 would then call up and download data from variousmemory locations whether they be memory locations on computer 140 ormemory locations I-com 154, I-ext 156 and map memory 158. The systemcompiles the entirety of the plaintext document by gathering thedispersed components thereof or compiles partial reconstructions fordifferent levels of security. By implementing different security levels,the system is dynamic enough such that server 152 can easily locate thevarious extracted data levels based upon various security codesrepresenting different security levels, as those codes are input by theuser at computer 140. Multiple security codes, at the inception andduring the process, may be utilized. The user may be required to inputsecurity codes at multiple times during the reconstruction orcompilation process. Regeneration of the source is possible withappropriate sec codes. Likewise, the source, via the processes herein,may be reorganized by using tags, labels and different named storagefacilities. Maps tracking locations of sec or SC stores enable the userto reclaim data, reform the data previously stored and reorganize thesame.

It should be noted that computer storage 154, 156 and 158 may be locatedon the same computer or may be located on different computers spreadthroughout the Internet. If the storage units are different computersspread throughout the Internet, computer storage 154, 156 and 158 wouldeach have their own URL or Uniform Resource Locator. On a LAN, thecomputer storage 154, 156 and 158 would each have their own addressesfor access thereon. In any event, during reconstruction, the server 152gathers the information and downloads the information into RAM 166 ofcomputer 140. This download may include a first download of the commonor remainder data from I-com 154. At a separate time, which may or maynot include a decryption routine, the extracted from I-ext 156 isdownloaded. Preferably, other than inputting initial security codes andany required or desired intermediate security codes, the system operatesautomatically without further input from the operator at client computer140. The download of both data sets may be simultaneous in that thedownload is not humanly perceivable. This is especially true if storagein different memory locations in PC 140 is utilized.

The role of server 152 may be expanded or reduced dependent upon thedesires of the user and the degree of security necessary. For example,server 152 may only enable separate storage of extracted data in I-ext156. In this limited role, server 152 would require the input of aproper security code and clearance prior to identifying and enabling thedownload of extracted data from I-ext 156.

In an expanded mode, server 152 may be involved in filtering the data,extracting the security sensitive (or select content) words, characters,icons or data objects to obtain extracted data and remainder datathereat, separately storing the extracted data from the remainder data(extracted data being placed in computer memory I-ext 156 and remainderdata being stored in common remainder data memory I-com 154) and thenpermitting reconstruction via separate or combined downloads of theremainder data and the extracted data into computer 140.

The innovation is a system and method for automatically or manuallycontrolled selection, extraction, storage, and release of selected andprioritized information. The system extracts selected information fromdata streams, in computers, computer networks communication devices, andnetworks, as well as electronic mail systems. The system and method canreside on a single computer, be distributed across multiple platforms,be distributed across multiple networks, or reside as a remote process(known as a hosted application service process in the state of the art).

B.4 INPUT OR INITIAL PROCESSING CONSIDERATIONS (BASIC FILTER MODULES)

The security sensitive (or select content) words, characters, icons ordata objects may be any word, phrase, letter, character, icon, dataobject (full or partial), image or whatever, as pre-defined or asestablished by the user. The user may specifically design the filter,begin with a dictionary (a content filter source) to define commonterms, identify any additional security sensitive words, letters,images, icon, data objects, partial versions of the foregoing or anyother granular aspect of the plaintext. After defining the filter andaccepting the data input, the system filters the plaintext and separatesextracted data (security sensitive (or select content) items) from theremainder data. The filter may also include elements of artificialintelligence (AI). For example, the user may select one word as asecurity word and the AI filter may automatically select all synonymouswords. A contextual filter may be thereby designed. The AI filter mayenable the user to define a filter in real time at the entry of data viaa keyboard. For example, the user may select to secure (i.e., extractand store) some proper names and may instruct the filter to secure namessuch as Block, Smythe and Cherry. During input of the plaintext, thesystem may detect Smith and ask the user if he or she wants to secure(a) all proper names in a common name dictionary collection and/or (b)all names with spellings similar to the filter input data, Block, Smytheand Cherry. As is known in the art, AI typically uses inference enginesto define one pathway or to outline a course of action. The filter orextraction engine discussed herein can be configured with AI, inferenceengines, neural network systems or other automatic systems to carry outthe functionality described herein for the dynamic operation of thesecurity system.

The system and methodology described herein also encompasses parsing theplain text document by bit count, word, word count, page count, linecount, paragraph count and parsing based upon any identifiable documentcharacteristic, capital letters, italics, underline, etc. Parsing is acontextual filter using a content as a marker and a range defines bycertain file parameters. Algorithms may be implemented to parse theplain text document. The target of the parsing algorithm (a bit count,word, letter, etc.) is equivalent to the “security word, character oricon, data object” (or SC) discussed herein. The parsing occurs with thefiltering of the plain text source document 100 and the subsequentstorage of extracted data apart from remainder data.

B.5 STORAGE

In a basic configuration, the common text or the remainder data isstored in common storage memory 108. The secure storage is similar tothe select content SC storage module. This common or remainder datastore is identified as A-com generally referring to a segmented memoryin a PC or a computer A in a network (LAN or WAN). It should beunderstood that reference to “remainder data” is simply a short-handrepresentation of data that is not extracted or filtered by the system.In some cases, remainder data may be nil and all content is removed.Accordingly, “remainder data” is simply that data which can be viewed,manipulated or further processed by the user inputting or initiallyprocessing the data. Remainder data storage 108 may include a confirmstorage signal function 111 to send back a confirm storage signal to thedata input device generating source plaintext document 100. Theextracted data file 106 is stored in a different memory computer storage110 (B-ext). In a preferred embodiment, memory segment 108 (A-com) is ata different location than computer storage memory segment 110 (B-ext).In a PC embodiment, memory A-com is a different memory segment thanmemory B-ext. In a networked embodiment, computer storage 108 may be ona different computer as compared with computer storage 110. In anInternet embodiment, common text or cleansed text storage is at one website (which may be one computer) and the extracted, high security datais stored at another web site, buried web page or otherInternet-accessible memory store location. In any event, the remaindertext is stored in a memory A-com and the extracted data or high securitywords, characters, icons or data objects are stored in memory B-ext.After storage of the extracted data in memory 110, a confirmationindicator 113 may be generated to the client computer or the computerhandling source plaintext input document 100 (the originating computersystem). Data may be stored on any type of computer memory or medium andmay include non-volatile memory, such as ROM, flash memory, floppy disk,disk drive memory, CD-ROM, and other permanent storage. Additionally,computer readable medium may include, for example, volatile storage suchas RAM, buffers, cache memory, and network circuits.

As a simple example, the program configured in accordance with thepresent invention, could automatically detect entry of all credit cardnumbers types into a user's computer. The filter is set to detect theunique credit card sequence and data string. Assuming that the user'scomputer is operating a browser and the user is communicating with aserver on the Internet, the user's computer would filter out the creditcard number and send the number to a secure storage site. The securestorage site is owned, operated or leased by a trusted party. Theextracted data, i.e., the credit card data, is stored at the trustedsite. The URL or other identifying data is sent to the vendor from whichthe user wants to purchase goods and services over the Internet. Whenthe vendor seeks to complete the transaction, the vendor sends a requestcode to the secure site, the trusted party at the secure extracted datastorage site debits the user's credit card account (or otherwise debitsthe user's bank account) and sends an approval code to the vendor. Inthis manner, the vendor is never given the user's credit card—the cardnumber is sent to a trusted party automatically by the filter in thesecurity program described herein. The security program may beincorporated in a browser to automatically protect credit card data,personal data (as a method to become anonymous on the Internet), etc.from being deliberately broadcast to others on the Internet or to blockothers from snooping into the user's personal data while the usercommunicates over the Internet.

In a further enhancement of the present invention, the computer or datainput device handling source plaintext document 100 may also record thelocation of A-com 108 and B-ext 110. The location data is called hereina “map.” A memory mapping function or module is utilized. The map may bestored in a third memory location 112. Memory location map 112 may be asegment of the memory of the data input computer originating plaintext100. The map may be encrypted for security reasons.

B.6 EXTRACTION AND STORAGE ENHANCEMENTS

As a further enhancement of the present invention, the user, prior toinitiating the security system, may be given a choice of filtering outall the uncommon words or words not found in the dictionary and addingcertain security sensitive (or select content) words, characters, iconsor data objects to filter 102. The added words or terms are filtered outwith the uncommon words. Of course, the user may be required to manuallyinput all security words or download the security word filter from theInternet or another system on the LAN. For security systems havingmultiple security levels, a plurality of filters would be created, eachfilter associated with a different security level. Further, multiplesecurity levels would require, in addition to remainder text document ordata 104, a plurality of extracted data documents 106. The common orremainder text document or data 104 would still be stored in remaindercomputer storage A-com 108. However, each extracted data document 106would be stored in a respective, separate computer memory segment orcomputer B-ext 110. Separate storage of a plurality of extracted data atmultiple, separate locations in B-ext is one of the many importantfeatures of the present invention.

The ability of the program to locate security sensitive (or selectcontent) words or characters can be enhanced by using a telephone book,properly dissected, to identify a collection of last names. Cities andtowns and street names can also be identified in this manner. Thecompilation of last names and cities, towns and streets can be used as alist of critical, security sensitive (or select content) words. Thefilter is represented by this compilation of words. Similar techniquesmay be used to create filters for scientific words, or words unique to acertain industry, or country.

In view of increasing levels of security relating to (a) the storagelocation A-com; (b) the transfer of remainder text document 104 tomemory computer storage A-com 108; (c) the storage of map 112 (possiblyencrypted); (d) the creation, storage or transfer of filter 102(possibly encrypted); (e) the storage of extracted data at memorystorage B-ext (whether singular or plural storage sites); and (f) thetransfer of extracted data thereto, the system may include an encryptione feature. The encryption e function 115, 117 and 118 isdiagrammatically illustrated in FIG. 4 .

The program of the present invention can be set to extract critical data(a) when the plaintext or the source document (data object) is created;(b) when the source document or data object is saved; (c) on a periodicbasis; (d) automatically; (e) per user command; (f) per ascertainable orprogrammable event; and (g) a combination of the foregoing. Timing forstorage of the extracted data is based on these aspects. Reconstructionof the data object or plaintext may be (a) automatic and substantiallytransparent to the user; (b) based upon manual entry of securityclearance data; (c) periodic; or (d) a combination of the foregoingdependent upon outside events and who is the author of the data objector other definable aspects of the data object, its environment oforigination, current and anticipated security threats and itsenvironment of proposed reconstruction. The timing for the extraction,storage and reconstruction is oftentimes dependent upon the level ofsecurity required by the user and/or his or her organization.

The system and method creates a high level of security by automaticselection and removal of critical and prioritized contents from a dataobjects stream, whether it be a digital document, digital file,database, sound bite, video clip, other structured, or streaming dataformats. The system and method enables a controlled release of theextracted data objects, enabling instant display of the instantaneousreturned contents, contingent on verification of user identity, accessrights, time of operation, location of source and or user, destinationof source and or user, and determine threat modes. The system and methoddelivers high security by removal of the selected prioritized contentfrom memories. The copies and traces of the selected extracted contentsare eradicated from the computer memory while the separated extract datastream is transferred to a safe removed storage memory media. Theextract, extracts, and any part thereof, will be return transferred touser display as soon as identity and access rights are validated.

A replacement of the extract (sometimes called a placeholder) can alsobe substituted on-the-fly to provide updated results, misinformation,dis-information, messages, alerts, links (to reports, data mining,search engines, web sites, and hyperlinks understood in the currentart), advertisements, and personalization and customization. Thevalidation can be done instantly by password, challenge questions andanswers, remote verification (phone, video, or personal contact withuser), or by biometrics means.

The extraction of data objects within data streams includes words,structured data objects, characters, numbers, bullet points, footnotes,prices, images, sound segments, video segments, and selected digitaldata packets. The extraction is conducted by separating a source(original) data stream into two or more extracts data streams. Thedifferent data object extractions are separated into groups reflectingpredefined contextual categories and restitution applications (such asto enable customization and personalization for the same or differentusers). The modified source (original) stream typically contains themajority of data objects of the source stream, whereas the extractstreams contains a minority of the data objects which represent selectedand categorized information and information deemed to be of prioritizedimportance.

The extracted categorized data objects are separated into one or morecontiguous data streams. The extracted data stream or streams arescattered to one or more storage memory memories. The extracted data canbe transported and shuttled between different storage or projectionapparatus, as directed automatically by various constraints including:security risk criteria of threats and attacks, sources, targets, users,policies, time of day, and threat modes.

The extracted data, in some cases, is transported to an online removablestorage and under extreme security threats to an off-line/off-network,digital or physical vaulted storage. Transport and shuttle is based onthe level of security alert. The use and release of the vaultedextractions is controlled by a set of rules or organizational policywhich includes the following options among others: (a) A vaulting ofsome, all, or specific data object extracts for long or short periods oftime. (b) Release of extractions into a display, in which the extracteddata objects will reconstitute with the modified original data stream,the original data objects stream or original document. (c) Release ofextractions into a projection display in order to project with themodified data stream, the original document while maintaining completeseparation of the modified source data stream (the source modified bythe extraction of data objects and insertion of placeholders) and theextracted data object streams. (d) Release of extractions into aprojection display in order to project a reconstitution of the originaldocument, in other words to create altered versions of the originaldocument, while maintaining complete separation of the modified datastream and the extracted streams. (e) In situations of high securitythreats, release of extractions into another projection display, PDA,floppy disk, paper document a wireless display, an overlay transparencywhile maintaining logical and physical separation of delivery streams.This will enable working with a representation of the initial source,but not the initial source itself, while understanding the prevalent,critical extracted information without comprising security to theinitial source material by exposing sensitive, identifying, or criticalinformation. (f) The distribution of sources, modified sources, orextracts to remote and distributed viewing devices. (g) Enable theongoing operation of information delivery and display in defiance ofknown ongoing or unknown security flaws, breaches, or eventscompromising the general state of security. (h) The delivery of distinctand separate data streams, delivered on the same or different channelsand media, each with minimal, limited, or even substantial usefulness inand by itself, that can be overlaid logically or physically toreconstitute the identifying data stream and display. Separate displaydevices can be used to create a composite image or they can be overlaidto physically separate display devices to reconstitute a usefulcomposite display.

The objective is to create security for the single computer or extendednetwork. When an intruder penetrates preexisting firewalls and othersecurity systems, the data object and streams, digital documents, anddigital files which will be valueless and prioritized data objectsrendered unidentifiable, the penetration is valueless because thecritical strategic information has been exported to a vaulted storage.Competitors or hackers, who learn that a computer or network isprotected by the system and method, might decide to attack anothertarget instead. This is comparable to a situation in which a bankrobber, who finds out that the bank vault is empty, will most probablylook for another bank.

The system and method has a menu of different options including theability to extract: (a) All existing databases on the computer ornetwork. (b) All newly loaded, mounted, or integrated data to thecomputer or network. (c) All plug-in memory devices (temporary orpermanent) containing data. (d) All new and imported data to thecomputer or network. (e) All new work and output created by the computeror network. (f) All data being transported in/out of the computer ornetwork including electronic mail. (g) All data being transmitted in/outof the computer or network including electronic mail.

The system and method releases the extracted data streams, subject to acontrolled-release mechanism and process. The release mechanism iscontingent on parameters including; rights to access specific contents,timing criteria, security restrictions, and preset policies. The releaseof the extracted data objects permits restitution of the source datastream in variations of the source that are full, partial, or modifiedrepresentations of that source data stream. The release provides forvarious levels (through user configuration) of separation between themodified source data stream and the extracted data streams. The systemenables the maximum grade of security by means of the option of a visualmerged projection of said different data streams, while maintaining astrict physical and logical separation between the data streams.

B.7 BASIC RECONSTRUCTION

FIG. 5 generally diagrammatically illustrates the major features of areconstruction routine (module) or system. The user, typically at acomputer terminal, inputs a reconstruction request 120. The system firstexecutes a security clearance protocol routine 122 in order to determinewhether the user has the proper security clearance. The securityclearance may be thought of as a security clearance control. If multipleusers are permitted access to the documents and those multiple usershave different security clearances, the security clearance protocoldetermines the level of security clearance and, hence, the full orpartial reconstruction of the plaintext. The security code input by theuser is checked against a security code database or list 124. Clearanceis provided in step 126. The location of the map and, hence, thelocation of the remainder data A-com 108 and extraction is provided tothe user's computer in step 128. This may include obtaining a copy ofthe map 130 showing the location of memory segments in (a) the localcomputer; (b) the LAN or WAN; or (c) the Internet storage sites. Thestorage segments are A-com 108 and B-ext 110. The common or remainderdata is downloaded or transferred or made available to the user'scomputer as shown at the output of map location and data step 128.Typically, the extracted or security sensitive (or select content) datafrom B-ext is downloaded. As described hereinafter, the data can bereconstructed as a complete electronic document in function 130 or maybe reconstructed only as a visual reconstruction in step 132. Visualreconstruction is discussed later. Function 130 operates as a compilerto gather the extracted data and remainder data into a single plaintextdocument. If the data object represents sound or audio signals,reconstruction and play back may require a speaker output in functionblock 130. In a telecommunications implementation of the presentinvention, the input would include a microphone or audio detector(supplemental to the input device for document 100), an analog todigital converter (possibly with a voice to digital converter), thefilter, extractor, storage facilities at least for the extracted data,and at the output of the system, a converter to audio and an audioannouncer. The recipient of the secured data stream or message would berequired to clear a security clearance and possibly obtain a decodingkey prior to listening to the entire, decoded message. The key and thesecurity data is separately downloaded to the recipient's device.

If remainder data in A-com memory 108 and extracted data in B-extcomputer memory 110 is encrypted, the reconstruction process includes adecryption step. Encryptors and decryptors are relatively well known bypersons of ordinary skill in the art. Further, the filter 102 (FIG. 4 )may include some encryption routine operating on the data object(plaintext) during the filtering. A simple encryption may includesubstituting “dummy” text or images for the security words and keeping apointer to an encryption key document mapping the security words withthe dummy words. The filter may be stored or may be destroyed at theoption of the user. Storage of the filter impacts the degree of securityof the entire data system but storage of the same filter enables theuser to reuse the filter at a later time. Encryption of the storedfilter increases the security of the data. Creation and storage of mapin memory 112 also impacts the degree of security of the system.However, if the filter 102 is destroyed and all copies of the map aredestroyed on the user's computer originating plaintext document data100, and the map is stored offsite in a third computer memory location112, this offsite map storage may enhance the degree of security of thedata. The originating computer processing plaintext 100 may be scrubbedto remove all reference and copies of the plaintext, remainder text,extracted data map storage data, etc., i.e., a deletion routine may beemployed on the data input computer.

B.8 RECONSTRUCTION TECHNIQUES

FIG. 7 diagrammatically illustrates a system diagram for variousreconstruction routines. A complete reconstruction is shown as securitylevel path A. This involves an electronic integration of plaintext instep 202 resulting from the complete electronic reconstruction ofdocument 100. For example, a merge may occur between the extracted dataand the remainder data or common text data. The document is completelycompiled in this process. Placeholders in the remainder document areutilized to locate and insert the extracted data. Most likely, therewill be no process controls imposed on the integrated document as shownin step 204. In other words, if the user at computer 140 has the propersecurity clearance, he or she could download or recreate the entireoriginal source, plaintext document and the user would be entitled toedit the document or change it in any way or copy it and reproduce it.

The second level of security, path B, results in storage of the commonor remainder data in a different memory location on the hard drive 168as compared with the extracted data. This is noted in step 206. Anotherwords, in a simple example, hard drive 168 or RAM 166 would hold a copyof a remainder data document and another copy of the extracted datadocument, that is, two separate documents. Since two documents areavailable in RAM 166 or hard drive 168, these documents are stored indifferent locations in the memory. In step 208, a map showing the memorylocation of the common or remainder document and the extracted datadocument is provided to computer 140. Step 210 commands the processorCPU 165 in computer 140 to interleave the extracted data with the commonor remainder data in the video board memory. In this process, theextracted data would typically have placeholders for the missingremainder data. Otherwise, control codes to locate the extracted datainto the remainder data would be executed by CPU 165 to properly placethe extracted data into the “visual space” of the placeholders in theremainder data document. The extracted data document may haveplaceholder for the remainder data. Some type of register between thetwo image documents may be necessary. The compiler, in this embodiment,gathers the document elements and visually compiles and presents theplaintext to the user.

FIG. 8 diagrammatically shows that video board memory 169 is loaded withremainder or common data 1 and a different location of the video memoryis loaded with extracted data 1. The next video memory location isloaded with common data 2 and then a different video memory location isloaded with extraction data 2. Since the refresh rate of computermonitor 163 is fast, the display 163 will show the common or theremainder data and then show the extracted data such that the user couldnot humanly perceive a difference in the document. However, the usercould not copy the document from display screen 163 (a “screen shot”)since the document is never electronically integrated into a singledocument. There is only a visual presentation of the combined documentby interleaving the extracted data with the common or remainder in thevideo memory 169. Step 212 notes that the user may be limited in his orher ability to process, edit and store the reconstructed and presentedplaintext document.

Security level path C recognizes in step 214 that the data is stored indifferent memory or computer locations. In this situation, two videoboards, video board A and video board B are shown as board 216 and 218.Video board 216 drives display monitor 220. Video board 218 drivesdisplay monitor 222. Display screens 220, 222 are overlaid atop eachother. Video board 216 is fed with common or remainder data from theremainder data store (see I-com store 154 in FIG. 6 ) and video board218 is fed with the extracted data from the extracted data store, forexample, I-ext store 156. In this manner, as noted in step 224, the useris presented only with a visual presentation or compilation of theplaintext. Since there was physical separation between video monitor 222and video monitor 220, there is no electronic integration at all of theplaintext document. Hence, the ability for the user to do anysignificant editing on the plaintext document is blocked or prohibitedbecause the user only has access to either the data on video board 216or the video board 218.

Security level path D shows that the extracted data may be parsed orfurther separated based on a plurality of security clearances in step226. Step 228 recognizes that the system can repeat process and securityprocess paths A, B and C only with portions of the extracted datapresented to the user based upon the user's security clearance.

C.0 TESTING THE SECURITY SYSTEM

The prime purpose of the data security system is to limit the disclosureof critical data or select content SC data to persons or organizationswho may misuse or abuse the data. A testing module supplies thisfunctionality to the infrastructure. With the advent of cheap memory,fast processors and increasingly dynamic search engines and the highspeed communications links established by the Internet, the ability of asuspect person or organization (a person/organization which is notpermitted to have the SC or secure data) to associate some low levelsecure or SC data with public source (or accessible private source) dataand “discover” the secret or SC presents a significant problem.Therefore, a system to test the secure nature of the SC or secure datasystem, with inference engines or knowledge expander KE search engines,is beneficial. The following sections C.1-C.23 discuss such searchengines to test the security of the base system described in sectionsB.1-B.8.

Also, the information infrastructure is flexible enough that the systemoperator (CIO) initially builds the infrastructure using simple filtersand simple access and permission polies and rules. The infrastructurebecomes more complex due to testing of the secure storage and the SCstorage. The higher degrees of complexity may cause the system operatorto employ more complex filter (see sections C.11, C.12 and C.13 andemploy aggressive deconstruction techniques (see section D.1, amongothers). Additional complexity is added to the system with furtherpolies or rules which are prioritized and implemented with hierarchicaltaxonomic classifications. The hierarchical taxonomic classes must bebought, customized or built. The knowledge expander KE module is used inthe development of the contextual filter modules, the testing modules,in the taxonomic filter modules and in the classification generator.

C.1 THE ETIOLOGY OF INFORMATION

Security, privacy and information sharing is predicated by therepresentation of the structure information. The structure has evolvedin usage over time from simple linear formats to complex hierarchicaltrees typified by tags, metadata and modifiers. Although the predominantinformation delivery and information storage format is in a linear datastream, the internal structure or representations include all possiblelinear, field defined, metric, tree, compound and combined layouts. Inother words, while data is delivered in a linear stream, the complexityof internal structure resolves into specific documented patterns,self-documenting meta data formats like HTML or XML, defined utilitarianand purpose-oriented formats like database management system (DBMS), ODF(open document format) or proprietary document object models (such asthe Microsoft DOM model). The combination and recombination of metadatain source documents or data streams complicates finding, location, andexpanding one's knowledge base of SC. The issue of the internal formatis important to the regulation, interpretation and application ofinformation.

As discussed above, the etiology of information involves a complexhierarchical tree of various types of data such as tags, metadata, andmodifiers. Specifically, to the issue of semiotic (words and objects inlanguage) and pragmatic (words relationship to user) meaning, theinternal format of information is important to its regulation,interpretation and the further use and application of the information. Adiscussion of the differentiation of the data as to content, context andconcept is discussed later herein. The abstraction of information isalso relevant here. For example, the database scheme DBMS and the DOMstyle sheets and the internal metadata all modify the encoding, format,structure, purpose and usage of the information. Links and referencesestablish contextual meaning, just as the environment establishesmeaning relevant to the multiple granular data in the information streamor document. Metadata can validate or invalidate meaning in the text ofthe data, for example, a format or a replace meta data element couldindicate the complete deletion of material, the connection between topicand footnote, or modification with non-usage of the data. To perceivedata only in context without the effects of format, purpose, referenceand access rights, potentially misinterprets the importance of contextand concept which may result in a misunderstanding of the information.

C.2 METASEARCH ENGINE FOR KNOWLEDGE DISCOVERY: SEARCH CONVERGENCE ANDDIVERGENCE

As an overview, the system's metasearch engine is a novel and usefulservice to uncover unknown or concealed relationships, delivery ofhigher relevancy search results and connecting the dots. Althoughmetasearch is derived from the technology for security and informationsharing, it is useful to augment standard Internet search enginesbecause it enables both search divergence for knowledge discovery andsearch convergence for assessing information integrity, the full processnecessary for connecting the dots. Search divergence extends search withaggregation, inference and data-to-data interaction beyond simplecontent into the realm of context and concept. The system is astandalone system or can be integrated to process results of othersearch engines. Presented is a process and a technology for knowledgediscovery, security and information sharing that is based on managementand control granular data content to secure information while alsoproviding interdomain information sharing. The basic assumption is thatall data/content must be monitored, analyzed and categorized at thegranular data level for basic security risk management and control. Riskmeasurement is driven by information recognition, multi-tier overlappinghierarchical meta-tagging technology, and granularization. This sametechnology, as a standalone deliverable product or service-orientedarchitecture, is applicable to knowledge discovery, uncovering unknownor concealed relationships, and for connecting the dots. The metasearchprocess is a serialized and repetitive process of knowledge discovery,usage of the innovation's secure engine, and the application of granulardata content, as a standalone service for knowledge discovery. Becausethe risk measurement is driven by (1) content, (2) context, and (3)concept, it is essential that the system recognize content not only inisolation to itself as a data stream or document but also its value whencombined with other private and public sources through aggregation,inference and data-to-data interaction. The metasearch is useful toaugment standard Internet search engines because it enables both: (1)search divergence for knowledge discovery and (2) search convergence forassessing information integrity. It completes the process necessary fordiscovering new knowledge to connect the dots. Although testing forsecurity is discussed above, testing for select content SC is possible.SC is unique information that is of some interest to the enterprise. Ifthe SC store is to be complete and represent a viable and usefulknowledge store, the quality of the information in the SC store can betested to ascertain whether the content is unique or should besupplemented as discussed later herein.

C.3 INITIAL PROCESS—METASEARCH ENGINE AND KNOWLEDGE EXPANDER

The automated metasearch KE search engine takes as source any datastream or document. The metasearch engine module (FIGS. 11, 12 a, forexample) automatically parses the source for content recognition andassessment of granular content. This granular content is automaticallymetatagged by the search engine—system for range (contextual relevancy),sensitivity level (prioritized), compartmentalization (sensitivitylevels at the same hierarchical level, but laterally or orthogonallydisposed to one another, see TS—Navy compared to TS—Army wherein theNavy and the Army are compartments), categories (hierarchical taxonomicclasses), relevancy (a type of sensitivity), and other multi-tieredoverlapping hierarchical factors. The granularized data content becomesthe search terms, while these metatags become metafilters for theknowledge discovery process. These search terms are submitted to anynumber of extant tools and services (other search engines, such asGoogle, Yahoo), for locating other relevant supplemental data streams,documents, databases, local, Intranet, Internet, and public or privatedata stores. Likely tools include Internet search engines, data miningtools, database query engines, data collections, indices and otherknowledge management (KM) applications or services, which are generallyclassified as data structures or data collections herein. Inductive anddeductive tools are advantageous too. Most tools can be easilyintegrated with the system, linked through service-orientedarchitectures (SOA), or their results piped into the source data steamor document (through Web 2.0 mashups). The metasearch engine also workswith standard—a priori—search terms or keywords, like any desktop,web-enabled, or Internet search tool. This search operation may bypassthe system's secure assessment of a relevant or representative sourcedata stream or document.

C.4 SIMPLE METASEARCH ENGINE

The results from the extant tools (the results sometimes calledsupplemental documents or supple docs) are logged for auditing,dispersed to maintain a legal chain-of-custody, and aggregated into asingle collated result. A List of Abbreviations is set forth in a latersection E.1. This collated result (the source, results, reports andlinks or references) is automatically submitted into the metasearchengine. Inputs include the configuration for either (1) searchconvergence or (2) search divergence. Search convergence conforms to thestandard purpose for using Internet search engines like Google or AltaVista because it defines, authenticates, and validates the search terms.The first level search results provide for an integrity check for theinitial information results in that it does not aggregate or inferencethe information therein.

The divergence function, on the other hand, is all about aggregation,inference, and data-to-data interaction because it specifically searchesfor links, references, relationships, outliers, and social networkingassociations to the search terms. The divergence function will likelyignore anything overlapping the search terms in order to expand thesearch. In other words, convergence increases the depth of knowledgewhere divergence increases the breadth of knowledge represented by thecollated result. The divergence function explores new, novel, unknown,and hidden connections. If you want to connect the dots, divergencedredges new but factually uncertain information where convergencethereafter authenticates.

The system is automatically run. The source (which is now the aggregatedresult from the extant tool processes) is automatically parsed by themetasearch engine with secondary recognition and assessment of granularcontent.

C.5 AUTOMATIC METATAGGING

This granular content of the source doc is automatically metatagged for:(1) Content categories, (name, location, date, dollar amount etc); (2)sensitivity level, (“Top Secret”, “Private”, “level 9”); (3)compartmentalization, (“top secret-Finance” “Top secret logistics”); and(4) relevancy, and other multi-tier hierarchical factors.

C.6 AUTOMATIC SECONDARY TAGGING OF A “RANGE”

A secondary level of metatagging may be created to define the “range” or“the area within the document” from which the keywords for the searchwill be selected. This is a simple contextual select content (SC) orsecure content (sec-con) operation. Software modules carry out thesefunctions. To create a higher level of relevancy in the search resultsthe system controls the granular data within the document. Only“areas”/“ranges” with a high relevancy will become the basis forlocating keywords for the next search. Range defines the area or areaswithin a document (characters from the target selected content, linesaway, within the same paragraph, on the same page, etc.), which will bethe base and platform for locating key words for the next search cycle.An analogy is the defining the range/area for oil drilling in a suburbof the city of Houston. All the drilling will take place only in thatlocation. As an example of defining a “range,” we present the system'screation of automated tear lines.

Each paragraph (the “range” can be also configured to a “sentence” or“line” or “page”) is automatically tagged based on an algorithm. Thealgorithm may be “tag the paragraph based on the highest sensitivitylevel assigned to any of its words; or if a group of specific wordsappear in a paragraph,” (“ATTA”, “Hamburg” “New York” all tagged asSecret “S” security level or importance) and tag the paragraph at ahigher sensitivity level than those words (i.e. Top Secret “TS”).

The granularized selected content becomes the search terms and themetatags become the metafilters for the systems meta search. Theautomated results of the meta search process is an augmentation ofexisting knowledge or a breakthrough to unknown or covert relationships.The convergence search vets the integrity of a priori search terms,while the divergence search generates new search terms and newreferences. The metasearch is repeated, as configured, for a fixednumber of times, for a fixed period of time, until results reach anasymptote, or indefinitely. The metasearch inputs also include themetafilters created prior to the presentation of the aggregated searchresult. These metafilters define the direction of the search byintroducing “fine adjustments” to the search process. In essence themetafilters narrow the scope of all subsequent and serial submissions tothe extant tools and services. The search may be configured to includeone or a combination of filters as follows:

-   -   (1) Selection of keywords for search based on their categories.        The system automatically categorizes each word and character. In        one embodiment, there are over 50 different categories or        hierarchical taxonomic classes. Categories may include name,        location, date, dollar amount, credit card number etc. As an        example the system may be configured to feed the search engine        with words that where automatically categorized by the system as        “locations” and “people” in such an example “Rome” “London” and        “Muhammad Atta” may be selected automatically as the keywords        for the next search cycle.    -   (2) Selection of keywords for search based on their sensitivity        level or important or assigned weight. This is the sensitivity        level of the hierarchical taxonomic classes. The system may be        configured to select as keywords for its next search only data        elements that where classified/tagged with a specific        sensitivity classification/tagging. A possible example, is an        instruction to use as keywords only words that where given “Top        Secret” classification, or “private” classification or assigned        with a specific weight (select words that where assigned weight        9 out of 10).    -   (3) Selection of keywords for search based on the specific        importance of their content. For example, use as keywords, words        that belong to a list of specific grouping of words. A list of        associated words that convey a concept or a contextual        relationship and other features and compartmentalization.        Metafilters for standard search terms, which bypass the initial        system's automatic review of a source, are optional.

C.7 MLS, MULTI-LEVEL SECURITY-COMPLIANT SEARCH AND DISTRIBUTIONS

Because the system and all the extant knowledge management tools may runin a secure system high environment, results are likely to be systemhigh too (classified at the same security level). However, the point ofany search or knowledge exploration is to share information anddistribute it to the users at the edge. The advantage of the system isthat the new search terms, the aggregate result, and all intermediatereports and documents are processed for Multi Level MLS-compliantsecurity and information sharing. Each user will get a result based onhis security level (for example, TS or S or C or UC). Specifically, thelist of new words, phrases, and other content is either automaticallyassessed or manually reviewed for addition to the system's dictionaries.Ownership sensitivity level, compartment, group, categories, mission,relevancy, and other multitier overlapping hierarchical metatags areapplied to each discovery and distributed to users subject to MLScompliance. The aggregate result is secured with multiple MLS-compliantversions and distributed to the edge on a: (1) per sensitivity-levelbasis version, or (2) base redacted document with objective securitycompliance for universal distribution and pushed to the edge with thesystem's reconstitution services. In other words, information sharing isimplemented either through the delivery of: (1) Multi LevelSecurity—MLS-compliant versions, or (2) Through a base redacted documentwith objective security compliance for universal distribution withreconstitution.

Reconstitution is supported in whole or in part as: (a) MLS-compliantresults, or as (b) Step-wise reconstruction, with defenses-in-depthMLS-compliant results i.e. controlled release layer by layer.

These two options are also applied to all intermediate results, reports,lists, linked or referenced sources. Of note, all links and references,even access to public data sources can be optionally restricted tominimize subsequent risks from aggregation, inference, and data-to-datainteraction. An agency with specific intent and knowledge is able to useaggregation, inference, and data-to-data interaction on public sourcesto create classified results. For example, if unclassified search termsincluding “president assassination British visit” returned newsclippings and RSS feeds of a forthcoming presidential visit to 10Downing Street, these public results are still unclassified but thecontext is clear and may be classified.

The systems metasearch is a novel and useful standalone service touncover unknown or concealed relationships, and connect the dots. It isuseful to augment standard Internet search engines because it enablesboth search divergence for knowledge discovery and search convergencefor assessing information integrity, the full process necessary forconnecting the dots. Search divergence extends search with aggregation,inference and data-to-data interaction beyond simple content into therealm of context and concept.

One of the main tools in the present knowledge expander (KE) engine isits ability to control granular data with automatic contentanalysis/filtering and tagging of all the select content SC data andunknown data elements in every document, data stream or input document.

The knowledge expander KE engine: (1) automatically creates a list ofsearch terms; (2) automatically cleans the noise from the search termlist; (3) automatically selects from the list of search terms those thatwill be sent to the next search cycle (preexisting rules for doing theselection)—example: select for search only the items found which are inthe “names category” and “locations category”—from the list of searchterms—for example the names “Redhouse,” “Kohn,” “Miami Beach,” and “FtLauderdale” will be sent to the next search cycle; (4) conduct adivergence search—limit set at 200 supplemental documents (supple doc).Each supple doc goes through an automatic content analysis/filtering andtagging of all the data elements in the document. Key words are locatedin the supple docs by filtering of categories. The process cleans noisefrom the extract key words from the supple docs. Rules select which keywords will be fed into next search cycle. For example, if 1 of the 200documents had the name “Bob Smith” location “Sarasota” (as long as therules for selection were not changed the next search will be fed withsearch terms—“Bob Smith” and “Sarasota”) search is expanding—a divergingsearch. The last process wherein Bob Smith is added to the KE engine isa convergence filtering process because the search results will convergeon Bob Smith.

C.8 BENEFITS OF THE METASEARCH—KNOWLEDGE EXPANDER

Typical search and data mining tools presupposes that the user alreadyknows part of the answer. The user must know how to phrase the question(search terns ir query) in order to get a proper answer. However, thisis only partial solution to knowledge management in that does not beginto address the real issues needed to connect the dots. This knowledgemanagement technique as called herein “convergent” because the operatorstarts with a known entity and the search confirms or denies the basicentity or gathers additional information in depth on the entity topic.The present knowledge expander search, with various modules operatingtogether, automates the convergent process until no additionalinformation is found on the topic.

The present system and process allows knowledge management as adivergent exploration. The basic starting point is any data stream (RSS,blog, documents, a fire hose of data) or multiple data streams, or acollection of search terms for an automated process. A single termdefines a manual process with a primary search term. The automatedprocess redacts the data stream(s) and looks for content and context ofimportance creating a list of primary search terms, metadata contextualfilters, and noise reduction tools in order to expand the breath ofknowledge with new links and references. Alternate spellings,misspellings, partial matches, duplicates, and other fuzzy technology isused to graft depth to a convergent search or prune a divergent search.In the case of a divergent search, the process is specifically exploringfor secondary search terms and information that is not part of theprimary search terms. Noise and duplicates are pruned. All hits andinformation located with URLs, X-links, or other pointers is retainedfor a chain-of-custody source-indicating data, reproducibility, andhuman intelligence analysis. Consolidated results are insufficient forlegal exploration, search warrants, or plans of action, so the retentionis critical to build a rationale for action and review therelationships, connections, and networks of people.

The search is extended with the primary search terms, the secondarysearch terms, and combinations and permutations thereof. Duplicates andnon-duplicates are pruned per the selection of convergence ordivergence. Tertiary search terms and information is aggregated in thesame manner as the secondary search terms and secondary information.This process would repeat unless manually interrupted, terminated by atimer, manually reviewed midstream and grafted or pruned, or terminatedby repetition or lack of new information. When no new information isfound both the convergent and divergent search are stopped. Thetermination rationale is inverted. In the case of convergence, thelatest results are only new information that is not wanted and adds nodepth to the knowledge. In the case of divergence, the latest resultsare only old information and represent no new breadth to the knowledge.

C.9 INFORMATION LIFE CYCLE ENGINE

The information life cycle engine has an input configuration which issaved, an indication of the source of the information and text,metadata, data streams, blogs, RSS (Release Simple Syndication or RichSite Summary), or a compound document. The process is a reduction of theinformation input into the system into its major elements by format,style, type and data type. Compound documents must be taken apart tobasic identified types with tags and metadata separated out.

FIG. 11 shows an input file 2001 (an information file or data stream)having therein text A, Text B, various tags relative to the informationin the text (such as paragraph numbers), metadata associated with itemsand data objects in the document and image elements. The input file issubjected to a reduction or deconstruction step 2002 which creates anexpanded information document which includes source data, and file mapof the various elements, text A, text B, metadata, tags, audio andimage. Of course, multiple audio files, image files, metadata files,tags etc. may be included in the expanded information documentrepresented at 2004. The Editor as an Information Expander section andfollowing sections provides greater detail of the reduction step 2002.

Further defining the source as an information file or data stream isimportant. “Text” is a basic defined type. The information life cycleengine automatically processes the information document. The processincludes selection, extraction and categorization. For example, adocument having security sensitive words or selected content (sel. cont.or SC in the figures), phrases, images or sounds, identified as securedcontent by Department of Defense pre-classification must be processedwith a step of filtering, identifying tear lines between classifiedmaterial (contextual filter or parsing technique) and ultimateidentification to a base level of classification. Words, phrases, userdefined words and a group list of words, phrases, combination andoccurrences within a defined range are employed to identify the security(priority) level of the information document being processed. Patternand categorization, dictionary categorization, and syntacticcategorization is employed.

The information document is also parsed or separated apart by syntax,that is, words, sentences, quotations, parenthesis, other types oftextual delineation, and instruction. Complex phrases are resolved foroverlapping security levels and complex meanings. Categorization isemployed to determine the level of security of the information.Multi-tier-overlapping and hierarchical tagging is employed. Any disputeautomatically identified by the system upgrades the entire informationdocument 2004 to the next highest security level. The next step ininformation life cycle engine is to extract and disperse variousversions of the document 2004 elements along tear lines or other typesof delineation specified by the operator.

Extracted data objects or elements may be replaced by tags, codes, ornull field indicators. The system may create various versions ofredacted output documents. The base document, redacted of all criticalor security sensitive information, should be stored along with arecovery file. The recovery file may include maps to permit a party witha preferred security clearance to recover completely the base ororiginal source document. A metadata table is created with anintermediate resultant document. Reports are generated showing encoding,dispersion of the data and storage location. Lists are also createdshowing selections of data redacted from the source document, encodingof the data and synopsis of the data. Additional outputs include arecovery file and a categorized meta search file. Displays are permittedeither locally or to test the discovery results.

FIG. 12 a shows information life cycle search module. At the beginningof the program flow in FIG. 12 a , an input function 2006 is noted whichincludes information regarding the source of the information document or“source document” which source doc may, at the operator's discretion,include search terms 2008, data stream 2010 or a source informationdocument file 2012. The search terms may include one or more SC orselect content or security sensitive content (sec-con) which is ofinterest to the user. If search terms are input, these terms aresupplied to a summation or discrimination function 2014 (AND, OR or NOTAND operator). In other words, the operator (or the system automaticallyby default) could conduct an AND conjunctive search obtaining additionalinformation from other documents or an OR exclusionary search showingand processing only the items that are identical are substantiallysimilar from source doc and filter outputs. The AND operation mayoperate on the terms applied to discrimination function 2014 (e.g., theterms from input 2008 must match one output of the simple filter 2018)or may refer to a summation of all terms from input 2008 and simplefilter 2018 and metasearch term engine 2020. The OR function has thesame options. The NOT AND discriminator operates only to extract and useterms output from metasearch term engine 2020 which do not match searchterms 2008 and the output of simple filter 2018. Some select content SC(or sec-con) must fall within the prioritized hierarchical taxonomicclass or classes function 2030 in the NOT AND search. The non-matchingsearch term engine is discussed later in connection with the ExpansiveUnknown Search Engine. A substantial similarity test, as a subroutine inthe discriminator 2014, may ignore duplicates and may truncate searchterms by eliminating suffixes (and possibly prefixes) from the searchterm processor. Input IN 2016 recognizes that the operator may provide amanual input into discrimination function 2014. If data stream 2010 oran information source file or document 2012 are input, these informationdocuments may be processed through one or more simple filters 2018extracting white list terms (inclusive lists) or black list terms(exclusive lists) or terms not found in dictionaries, the result ofwhich is supplied to the summation-differentiation function 2014. Seethe Simple Filter and Complex Filter sections C.12, C.13 below. Inaddition, data stream 2010 and document source file 2012 are submittedto a metasearch term engine 2020. The more highly organized metasearchterm engine 2020 filters the data stream or source document with acontextual filter 2022, a conceptual filter 2024, a sensitivity levelfilter 2026, a hierarchical taxonomic or category analysis 2030 and aparsing algorithm for word count, line, paragraph, or DOL 2032(contextual filter with range setting module). The order of thesemetasearch filters may be changed. The filter modules are contextual,semiotic and taxonomic filters. Semiotic include syntactic, semantic andpragmatic features (discussed later in section C.13). Sensitivity level2026 contemplates potential manual input IN 2028 which establishes thedepth or expansive nature of either the contextual filter or conceptualfilter. The sensitivity filter and the hierarchical taxonomic filtersare used to focus the search in that the operator can selecthierarchical taxonomic levels or terms of concern, for example, names ofterrorists—priority 1, terrorist location—priority 2, date of sourcedoc—one week—priority 3, temporal (date)—1 month—priority 4, terroristorganization—priority 5. Each “priority” is a SC selection sensitivityfactor (or sec-con factor) and the class of the information is ataxonomic analysis, the result of which is a prioritized hierarchicaltaxonomic classification system and filter for the search. The result ofthis meta or more highly organized search term engine 2020 is suppliedto the summation-discrimination function 2014.

The system then operates on convergent or divergent function 2034 whichincludes, optionally, input IN 2036 from an operator. The convergent ordivergent function 2034 enables the operator to only search forconverging search terms or select a more divergent search routine.Function 2038 results in a primary set of search terms.

An element of the information life cycle engine is parsing ordeconstructing the original source document or information file. Thestructure of the source document DOM metadata and compound file formatmust be simplified and the source document must be broken into itsatomic types such as markup data, tags, metadata, links, hyperlinks,references, comment, differing data types, purpose and format. Thisparsing is discussed later in connection with the DOM data structure insection C.18, among others. Categorization is applied as a resolution orcoding to a single overriding security level. Priority is given to themost complex or highest security level. For example, a document with“Bin Laden” as compared with “Bin” next to “Laden” results in “Bin”being identified at the secret “S” level and “Laden” being identified atthe classified “C” level wherein “Bin Laden” is classified at the topsecret “TS” level. (The security classes including TS top secret, Ssecret, C classified and UC unclassified). The resulting document wouldbe identified as top secret TS because “Bin Laden” represents thehighest level in that paragraph (format or contextual indicator).Adjacent permutations of a linear nature could also be employed. Forexample, “Khalid Sheik Mohamed of Syria” results in unknown [Khalid],Sheik [title of middle eastern person], and Mohamed [name] “of”[preposition] and “Syria” [geographic territory]. The resulting analysisresults in an up coding of categorization to a next higher securitylevel (e.g., S to TS) due to the contextual range grouping of the SC.

The system may employ a “My Group” list or profile which identifies keyitems and triggers a certain result based upon the “My Group” pre-setprofile list. The profile is an SC profile of interest or a sec-conprofile of interest. The My Group list would include a profile name suchas “first strike capacity” and a list of key terms under that profilesuch as “nuclear, ballistic, submarine” and a “range” such that anytimethat key word is found in a paragraph, a certain classification isassigned. The “range” being the paragraph in the source document. TheProfile Name could be used as a search term for a divergence search(discussed later) or a link to a dictionary of search terms. The searchterms are either content-derived in nature or conceptual in nature.Certain contextually-derived results, see Bin Laden example above, mayautomatically trigger use of a specially designated My Group searchprofile. Further, the Profile Name can be linked to another list ofsynonyms associated with the specific terms nuclear, ballistic,submarine. Contextual and conceptual terms are employed in this manner.

C.10 INFORMATION LIFE CYCLE SEARCHES

The input into the information life cycle search (FIG. 12 a ) may be oneor more search terms, a data stream such as a blog, RSS, or a string ofdata, or information document (DOM). FIG. 12 a graphically shows theprocedure for the information life cycle search. The systempre-processes the input and obtains metasearch terms both in acontextual manner and conceptual manner. These are identified withfilters discussed later on. Sensitivity levels are set and theinformation is categorized both in a contextual manner and a conceptualmanner. Ranges of data representing format choices are used such aslines, sentences, DOL's (lines of data) and paragraphs. Thereafter, theinformation life cycle search engine modifies the search terms and theuser inputs either a convergent instruction or a divergent instruction.The resulting search terms are then supplemented according to theconvergent or divergent system.

The search for the primary search terms (convergent or divergent)results in a federated or confederated as well as a distributedsupplemental search term group. Hyperlinks, URL, network references, SQLare submitted to one or more of the following: search engines,databases, data warehouses, addressable data elements, artificialintelligence, data mining sources, text storage, method data storage,indexes, libraries, catalogs as well as other data structures.

The next step involves ascertaining the depth of the search byidentifying the number of hits for the first search term and the lastsearch term, the time involved to compile the search, the time involveduntil no changes, the number of iterations involved as based upon adivergence search as separate from a convergence search. The output is aconsolidated search result list.

FIG. 12 b starts with an input being primary search terms 2038. Thesearch engine module then executes function 2040 which is either afederated, consolidated, or distributed search. A federated searchunites the search terms and additional supplemental documents generatedand located from those search terms together. A consolidated search issimply a composite of all the search documents obtained without anyrelationship or affiliation. A distributed search seeks to widely engagea large number of data structures for the search. Operator input IN 2042directs the selection of the search function 2040. Search function 2040operates by sending the primary search terms to additional datacollection targets such as search engines 2044 (Google, Yahoo, etc.),databases 2046 (representing a plurality of databases), data warehousessuch as dictionaries or other sources 2048, addressable data structures2049, artificial intelligence or mining operation functions 2050 andother data collections, data structures, indices, etc. The miningoperation 2050 may access further tertiary data structures 2052. Thesearch function 2040 applies these primary search terms to text stores2054, metadata stores 2056, indexes 2058, libraries of words, terms,images, data or data objects 2060 and catalogs 2062. The results arecompiled back to search function 2040. Step 2064 is a depth or searchdepth control in which the operator by input IN 2066 delineates eitheras a pre-set or a default or an active real-time control the number ofhits needed from each of the data structures accessed by search function2040, the amount of time for the search, the amount of time when nochange is found from the retrieved documents, the number of iterations(repetitions), and whether the primary goal is a divergent search or aconvergent search. Search depth control 2064 provides controlinformation CNTLR 2068 back to search function 2040. The search results2070 (supplemental documents or supple docs) are obtained either fromoutput of search 2040 or after the depth control 2064 is applied to thesearch.

The third step is to take the secondary search result or theconsolidated search result and apply them in a convergent manner ordivergent manner. The output is either a convergent compilation ofdocuments which are obtained via the supplemental search terms (addingknown SC to the search term group) or a divergent compilation ofdocuments employing the same search terms. A similar output result canbe obtained by using hyperlinks, URLs and other metadata aspects.

The fourth step (see FIGS. 12 c and 12 d ) is to build a tree or a tablelisting the details. The tree shows the hierarchy of the data retrievedin the convergent or divergent search and the table lists the items suchas profile, time, username, link, context, etc. FIG. 12 c shows that theprimary search term generates a certain supplemental group of documentswhereas the secondary search results in a different group of documentsand the tertiary search generates a third level group of documents. Thenumber of searches is keyed to the time involved and the criticality ofthe information sought to be searched. The object of the tree buildingand table building (FIGS. 12 c, 12 d ) is an exploration of informationlinked or associated with the initial information provided. Further, asocial relationship or connect-the-dots matrix display can be obtainedbased upon this iterative information life cycle search.

The output or yield of the information life cycle engine module resultsin a convergence search wherein additional supplemental information datais obtained to corroborate or validate the initial search terms andinitial information document supplied. Therefore, the sec-con is testedor the SC is tested in the respective storage mediums. A divergentsearch yields more data which is loosely associated or related, coupledor referenced to the initial information document. Initial search termsresult in a divergent document can be expanded upon by the hierarchicaltree with a list of details.

At some point, the operator must prune the hierarchical tree, reduceunnecessary items and augment others. Some of this pruning can beautomated with an inference engine or artificial intelligence module.Otherwise, the operator may manually engage in the information lifecycle engine.

The last step is a relationship map between the initial document and thesupplemental documents generated through the convergent or divergentsearch. The final result is a map of the hierarchical search terms atvarious levels source, search 1, search 2, search 3 . . . search n. SeeFIG. 12 c . A further result is shown in FIG. 12 d wherein a socialrelationship or process relationship between the various documents isestablished. Some people would call this connecting the dots orrelationship mapping. The relationship mapping changes the degree ofseparation or similarity between each information document.

FIG. 12 c begins with the primary search results 2070 and function step2072 executes a convergent test and a divergent test on the primarysearch result. The basis 2074 could either be the search resultsthemselves, a search term generation which is search terms extractedfrom the documents located during the search, hyperlink or x-link andURLs. The convergent test determines whether the search and recovereddocuments converged into a certain pattern whereas a divergent testshows that the documents, search results, hyperlinks or whatever spreadto wider results. The build and display function 2076 takes the primarysearch terms and builds a search tree and builds a table. Operator inputIN 2078 optionally enables the operator to set the scope and format ofthe search tree and the table. Tree 2080 is a display showing theprimary search terms n, n+1, n+2 and secondary search terms m, m+1, m+2and tertiary search terms o, o+1, o+2, etc. Iterative search function2084 establishes how many paths or branches are executed in building theprimary search tree. This may be one of the input IN 2078 functions forbuild and display function 2076. The iterative search 2084 may be anumber of times counted in branches or may be certain time unit g, h, i,j. Table 2082 is a detail showing the search term including profile,time, username, link and context of the search term. Function 2086 isthis first level output report showing the search terms for theinformation life cycle engine.

FIG. 12 d begins with the first level output for the search term 2086.Function 2090 is a convergent display which highlights the same, similaror closely related documents. Also, an output ratio 2092 may show howmany of the documents are related by word, term, image segment,hyperlink or metadata factors. Step 2094 is a divergent display which isthe opposite of the convergent display. Output factors 2096 may show howmany divergent documents having little or no connection to the primarysearch terms have been discovered by the search engine. Function 2098permits the operator by input IN 2100 to truncate the search orsupplement the search. Function 2102 repeats the search on command ofthe operator. Step 2104 categorizes the search results by key terms(taxonomic classification) and generates a map. One map is shown by tree2106 which shows the search results or information documents as documento, o+1, o+2 as well as secondary documents p, p+1. Step 2108 develops arelationship map with associated terms, phrases, documents orhyperlinks. Relational tree 2110 shows that point or document p+1 isrelated to document o+1, p and o+2. In contrast, document o is notrelated to any document other than o+1. The relationship tree 2110 issometimes called a “connect-the-dots” map.

C.11 SEARCH WITH PRIORITIZED CLASSIFICATIONS

FIG. 12 a shows that the operator or user can input search terms, sourcedocument or a data stream 2006 into the system. Search terms function2008 represents the use of a search term string, document or data streaminto the metasearch engine 2020 and through the classification orcategory analysis filter (hierarchical taxonomic system) function 2030.The sensitivity function 2026 permits the operator or user to pre-selectthe priority for the hierarchical taxonomic or class system.Alternatively, “My Profile” or pre-set prioritized classes may be used.The user may also select the priority and the class/subclass of thesearch terms immediately prior to the search. The process, in FIG. 12 a, extracts search terms from input data represented by a source documentor a data stream, and identifies data elements in the input data withthe classification system corresponding to n priorities therein. In oneembodiment, the operator selects either a convergent search or adivergent search in function 2034. Optionally, the convergent—divergentsearch function 2034 may be omitted. The process then applies theprioritized and classified search terms as primary search terms in FIG.12 b , term function 2038, gathers documents and then resets npriorities in said classification system to m priorities in saidclassification system. This is part of the depth control function 2064and control function 2068. Secondary search terms from the input dataelements (terms 2008), classified corresponding to m priorities areagain used in function 2040 and the search is repeated to gathersecondary supplemental documents. The system then appliesconvergent—divergent test function 2072 (FIG. 12 c ) on all supplementaldocuments to find convergent or divergent characteristics of thegathered documents. The search repeats until a search end parameter ismet, such as all supplemental documents exceed a predetermined number, apredetermined number of data elements from all supplemental documentsfulfill a predetermined number of priorities in said classificationsystem, a lapse of a predetermined time, a predetermined number of dataelements from all supplemental documents fulfill a predetermined numberof priorities in said classification system, among other search endparameters discussed herein.

Optionally, the system and process may omit convergence—divergencefunction input 2034, 2036, and select n priorities from the prioritizedhierarchical classification system, extract search terms from input data(functions 2006, 2008, 2010, 2012) represented by a source document or adata stream or a search term string, and identify data elements in saidinput data with said classification system corresponding to the npriorities. The process then searches data collections with said searchterms and gathering supplemental documents based thereon. FIG. 12 b .The process then extracts secondary search terms as data elements fromthe supplemental documents with the classification system correspondingto n priorities and repeats the search of data collections with thesecondary search terms. Secondary supplemental documents are thengathered. See FIG. 12 d , truncate, supplement and search function 2098,and repeat search function 2102. Such an optional search ends asdescribed above.

C.12 SIMPLE FILTERS

The user, prior to initiating the knowledge expander select content (SC)engine module, may be given a choice of filtering out or identifying allSC data objects or sec-con objects with white lists or black lists or acombination thereof as a content filter. Uncommon words, terms or dataelements not found in the dictionary, geographic term lists or namelists and other lists are located with a negative content filter used todiscover unknown data elements. The negative content filters can beadded to the metasearch term engine such that these unknown words, termsor data elements are tagged “unknown” and assigned to the hierarchicaltaxonomic class as an unknown data element at the hierarchical taxonomicclassification level (assigned to a class of words, terms, etc., andassigned an SC sensitivity level—that is—a hierarchical marker). Thesearch terms extracted from the source doc are supplemented with whitelists or black lists words, terms, etc. (content filters additions) andthe supplemented SC or select content words or terms are filtered out toidentify the uncommon or unknown words. Of course, the user may berequired to manually input all SC words or download the SC word filterfrom the Internet or another secure network system or LAN. A “profile”of SC with the hierarchical taxonomic markers can be employed by theuser-operator. For select content systems having multiple levels ofimportance (which may be multiple security levels or multiple levels ofSC importance in an organization or related to the organization, forexample, primary competitors compared with secondary competitors), aplurality of filters would be created, each filter associated with adifferent SC level. Further, multiple SC levels may require, remainderSC document or data stores (unknown data stores) and a plurality ofextracted data stores. Multiple levels are similar to the MLS securitycontent (sec-con) system described in section D.4 to D.9.

The ability of the program to locate select content or SC words orcharacters (or sec-con) can be enhanced by using a telephone book,properly dissected, to identify a collection of last names. Cities andtowns and street names can also be identified in this manner. Thecompilation of last names and cities, towns and streets can be used as alist of critical, SC words. The filter is represented by thiscompilation of words. Similar techniques may be used to create filtersfor scientific words, or words unique to a certain industry, or country.

C.13 COMPLEX FILTERS

There is a need to construct filters which supplement the initial listor compilation of SC (or sec-con) words, characters, icons and dataobjects (herein “word” or “data object” or “word/object”). The needarises either due to the fact that the initial SC search termword/object list is incomplete, or that the author of the initial listis concerned that the list is too limited or in order to defeat anattack or an inference engine “reverse engineering” at the resultantexpanded SC document. In a test mode, the complex filter is used todetermine how “secret” or “special” the sec-con or SC is compared tooutside databases and data collections. Further, the incorporation of afilter generator for SC or sec-con search terms enhances the userfriendliness of the program. In one embodiment, the program isconfigured as an editor compiler to screen and build enhanced SC doc ordoc collection from a source document. The user selects, at his option,functional aspects which include: compliance with laws (an applicationof a type of filter, e.g. HIPAA, GLB, Oxley-Sarbanes, EU privacy,executive orders); privacy (another type of filter which locates SCterms, for example, social security numbers, see also, EU policy);search for and supplement filter; pay per view (which enables the userto buy missing sensitive information (for commercial purposes); survival(which creates a distributed and dispersed copy of the user's sourcedocument and other stored documents and items using predeterminedstorage facilities); security (which triggers the various securityroutines); and storing (which permits the user to select which of theseveral storage options the extracted SC data/objects should be employedin the dispersal.

The filter routine or module diagrammatically illustrated in FIG. 13 isuseful in compiling a SC or sec-con search term filter which separatesboth the sensitive word/objects and contextual and semiotic andtaxonomic aspects of the initial list of SC word/objects. The filterworks in conjunction with a compilation of data, typically located on anetwork which could be private or public. In low level SC situations,the filter may access Internet databases to gather additional data forthe filter. In more robust SC systems, the filter could access privatedata bases (one located at the same organization level as the user) andbuild or compile the additional SC word/objects. The filter program 950in FIG. 13 begins with step 952 which compiles the initial list of SCword/objects. In 954, the initial list is supplemented withdictionaries, phone books, corporate records (to obtain subsidiary dataand trade names) and thesaurus data. This is a content filter. Each ofthese represent different compilations of data and the added data isadded to the initial list of SC word/objects. In 956 a search isconducted on a network, usually through a search engine, to gatherexcerpts near and abut the keywords. This is a range or contextualfiltering aspect. These keywords are the initial SC word/objects.Statistical algorithms are applied to gather non-common word/objectswhich are associated with the keywords as found in the additional datacompilations. The goal of the adaptive complex filter is to obtaincontextual, semiotic and taxonomic words, characters or data objectsfrom the compilation of additional data related to the SC words,characters or data objects. Semiotic is a general philosophical theoryof signs and symbols (read language and words and objects) thatespecially deals with their function. Semiotics include syntactics,semantics and pragmatics. Syntactics is the formal relationship betweensigns. Semantics is the meaning of signs and pragmatics is therelationship between signs and their users, such as the relationship ofsentences to their environment. Taxonomy is the scientificclassification and categorization of items. Therefore as an example, asearch through the Internet on Google search engine under “Bin Laden”may show a number of uncommon (non-dictionary words)(contentfilter-based search) within 200 words of the target “Bin Laden” (acontextual filter-based search). This search string would gatherdocuments from the Google search and copy 200 words on either side of“Bin Laden” and then extract only non-dictionary words into asupplemental SC term list. This type of filter algorithm looks forcontextual matters close or near to the target. The search is semioticand statistical in nature. Additionally, the initial supplemental listwould identify the Bin Laden is an Arab and this classification (ataxonomic aspect) can be used to expand the list for the filter. Thealgorithm may include a simple command to gather all 10 words on eitherside of Bin Laden. This is a pure contextual search and the “10 word”range or format aspect is a statistical number. From the supplementallist, all pronouns, prepositions and conjunctions may be eliminated.Spiders or robots may be used in the gathering of the contextual andsemiotic filter data. The contextual, semiotic and taxonomic words,characters or data objects from the compilation of additional data areall related to the initial list of SC words, characters or data objects.

Step 958 compiles the adaptive or complex filter. The above notedcontextual, semiotic and taxonomic filter is adaptive since it can beused to expand (and potentially contract or reduce) and adapt anexisting list of SC or sec-con word/objects to a larger list whichbetter expands the information content of the source document andsupplies SC term data to an inference engine. As a test, the sec-constorage is tested with the complex filter to determine whether thesecurity level is appropriate. For an SC test, the complex filter isused to ascertain whether the SC store should be supplemented withadditional outside data or that the H-tax class be changed. Step 959repeats the filter gathering and compilation for various levels of SCinquiries. Higher SC inquiries require a broader search (1000 uncommonwords near Bin Laden and add all Arabic and sub-Asian continent cities).Orthogonal SC groups (those groups having the same level, e.g. S-Secret,with each other but being different organizations, e.g, Department ofDefense compared to the FBI) often have different methods to keep SCdata separate between compartments.

The adaptive filter can be set to automatically gather additive SCword/objects. The test module may be run periodically. The system, witha basic filter, may identify a SC word in a paragraph being scanned bythe initial filter. This SC or sec-con word may be a special word in theexisting filter or may be a non-common word not found in the initialfilter. The adaptive filter system may then obtain this “unknown” or“special” word (a negative filter in that the word-object is not matchedto a word-object having the same content), and conduct a search througha compilation or data base of additional words, etc. Any newword/objects falling within the contextual, semiotic and taxonomic SCwords, characters or data objects from the compilation of additionaldata (database) related to said SC words, characters or data objects arethen added to the filter. The expanded filter is then used to supplementthe source document.

Step 960 compiles a supplemental filter with random words, phrases, etc.in order to further defeat an inference engine reverse engineeringassault on the SC supplement document matrix. In some sense, theproduction and use of a random filter is an encryption technique sincethe resultant filtered product, in order to be understood by others,must be reverse filtered or decrypted to reveal the source doc andenhanced doc matrix at the appropriate SC level. Nonsense words may beadded to this supplemental filter. Step 962 applies the primary filter(with the SC word/objects and the additive word/objects from thecontextual et al. filter) to the source document. Step 964 extracts theSC word/objects per SC organizational level. It is noted that severalfilters are used, on one for each SC level, whether hierarchical ororthogonal. The extracted SC word/objects are stored as a supplementalsearch term doc and the system gathers supplemental documents (suppledocs) which expand the knowledge base about the SC word/object. Step 966applies the supplemental filter to the supple docs returned per searchlevel with SC search terms. The system then repeats the process of SCsearch term generation on primary retrieved supple docs level 1,generates SC search terms per level 2, and retrieves SC level 2 suppledocs. A tertiary search term generation and supple doc retrieval ispossible based upon the operator's initial set-up or upon operator inputat steps 964, 970. Step 968 stores the supplemental doc to permitinformation enhancement of the source doc. Step 970 publishes,distributes or pushes the source and all supple docs and search termdata to others having a need to know.

C.14 THE EDITOR AS AN INFORMATION EXPANDER

FIGS. 14 through 18 diagrammatically illustrate an editor which may beemployed to locate SC or sec-con word/objects in a source document andexpand the knowledge base with supple docs. In one embodiment, theeditor is a standalone application or a module to add onto otherapplications for plain text and media creation, editing, and sensitivitySC level tagging. Other types of tagging, wherein the editor supplementsthe initial group or subset of select content SC or sec-con sensitivewords, characters, icons and data objects by categorization, taxonomyclassification, privacy, security, compliance, and semiotic meaning, arealso available. The editor supports a full range of document managementand can be integrated into a unified infrastructure, from creation,editing, document markup, tagging, tag conversion, tag removal, contextsensitivity level redaction, context reconstitution, support for complexprocess work flows, and expanding the knowledge base by addingsupplemental documents (supple docs) to the initial collection of sourcedoc. The architecture assures separation of data from metadata so thatno security lapses are introduced into the traditional word processingand document management cycle.

The Editor automatically and with nominal operator input (after theEditor is initialized) separates the data stream from all markup andtagging word/objects for SC knowledge generation purposes.

The interlacing of user content with metadata creates significantprocess, storage, distribution, and workflow security failures that arenot resolved with current technologies. Current technologies includeencryption, firewalls, intrusion detection, perimeter guards, and lockeddistribution packages.

The Editor enables text and media creation. However, all additions,deletions, changes, insertions, and reorganizations and reordering aretracked as metadata that does not become part of the document orinformation file. The document (information file) as seen and shown tothe user represents the deliverable format. Since formatting ismetadata, it is not included in the representation. Formatting, suchfont sizing, colors, font selection, footnotes, headers, subscripts,superscripts, line numbering, indexing, and other featurescharacteristic of standard document preparation can be supported but arerepresented only as metadata. Tagging, including SC sensitivity level,categorization, taxonomy classification, privacy, security, compliance,and semiotic meaning are also represented only as metadata. Thisseparation of representation from meta-representation is critical forcreating the infrastructure for SC knowledge expansion, secureinformation sharing, privacy, security, and compliance.

The editor is currently set in a WINDOWS environment. Pulldown menusprovide access to formatting and tagging features. The document, fromsource, precursor (marked and tagged but not yet filtered or extracted)and resultant final versions for each SC sensitivity level, as seen andrepresented to the user, is distributed in resultant final form, therebyassuring SC knowledge level compliance. No hierarchical, hidden,encapsulated, linked, associated, or referential information is part ofthe data stream, file, or storage.

Metadata (such as formatting, such font sizing, colors, font selection,footnotes, headers, subscripts, superscripts, line numbering, indexing,and other features characteristic of standard document preparation) isusually hidden from the user. This supplemental metadata informationcontains all markup, tagging, formatting, and process supportinformation for the editing process and enables immediate granulardistribution of the data stream subject to the needed SC compliancerules. In other words, the data stream can be automatically processedwith other functions to satisfy multiple competing requirements and SCsensitivity levels.

FIGS. 14, 15, 16, 17 and 18 are discussed concurrently herein. FIG. 14is a basic flow chart for one embodiment of the Editor. Editor program972 begins with obtaining the source document 974. Of course, the sourcedocument may be any type of document or may be a data stream. Of course,the data stream is typically delimited by start and stop characters orcodes. Hence, the term “data stream” is similar to “source document”herein and in the claims. Step or function 976 obtains one or morefilters for one or more SC or security sec-con or sensitivity levels.Step 978 screens or processed the source document with the filter(s).For example, the source document in FIG. 15 in window 991 has textregions 993, 994, 995 and 996. In step 979, the Editor displays, in situ(in the displayed document), the filtered identified SC or sec-conmaterial and conforms the precursor document to the SC or sec-consensitivity protocols. As shown, the SC or sec-con sensitivity protocolsare security level protocols for the system within which the Editor isemployed as an information processing tool. SC or sec-con sensitivitylevels 1, 2, 3, etc. correlate to security levels TS, S, C and UC fortop secret, secret, classified and unclassified. FIG. 16 shows that theaddress data 993 is marked TS (top secret), region 994 is displayed incolor A for TS coding (please note that the addressee data may also beso marked) and is “red-lined” or struck out. In an information-expansionsystem, the process marks, labels or tags each word. This is a contentfilter and tag regime. The user may manually upgrade or downgrade SC orsec-con levels. Region 995 is displayed as presented in the sourcedocument and is labeled U (unclassified) and region 996 is shown incolor B, is redlined and is labeled S. Labels TS, S, C (classified) andU are the established security labeling protocol used by theorganization employing the Editor. The same tagging for each word orcharacter in the source doc can be used. Other labeling schemes for SCelements may be employed. Color is used to assist the user to select(and in some non-standard cases, deselect) the SC or sec-con data markedby the editor. Redline is used to inform the user that the filter(s)will tag and extract the marked data. Labels are used to permit theentity using the editor to employ standard tear line protocol. Tear lineprotocol is a contextual tool. Any data beneath a securityclassification of the user is under the tear line and the data ispermitted to be distributed to the lower security cleared user. Ofcourse, electronic distribution of secure data need not use the hardcopy or print version of the tear line. However, this nomenclaturereferring to the tear line is used in the prior art systems.

Step 980 accepts the user's manual changes (typically upgrades) to theprecursor document. These manual changes are displayed, redlined,colored and labeled. Step 982 inserts the SC or sec-con tags (orsecurity label TS, S, C and U for the sample doc in FIGS. 15, 16, 17 and18 ) as discussed above. Step 984 notes that the system takes certainmetadata such as author, date-time, version history, change history,etc. and converts this meta data into ordinary text, marks that data atthe necessary security level or SC or sec-con sensitivity level andlabels the meta data. Step 986 permits the user to add (or omit)placeholders into the final document. FIG. 17 shows placeholders asblack lines or as XXXXX symbols (or other symbols) wherein the sensitiveSC or sec-con text is not shown but some replacement markers are shown.The byline in region 1003 show “sanitized document.” The byline 1003 inFIG. 16 lists the sec-con security level (or SC level of priority) andthe color representation.

In a specific knowledge expander engine, each word, term and characteris labeled or tagged with a content filter to show data object type(name, town, credit card number, etc.) and an SC or sec-con sensitivitylevel (a “tax” or taxonomic classification and sensitivity level). Seestep 982. The user may add tags (step 979, 980). Meta data is alsotagged or labeled.

Step 988 activates the filter, extracts the sensitive SC or sec-con dataand temporarily stores the extracted SC or sec-con data. The expansiveknowledge engine also stores negative filter results which are thosewords, terms or characters, images, not found in the context filters.Step 990 displays the filtered document and the user may view thefiltered document at each SC or sec-con level. Therefore, the user,before transmitting a secured email (or letter) doc may look at thesource (FIG. 15 ), may look at the TS level (FIG. 16 ) without theredline strike out but with security labels and colors, may look at theT level revealing regions 996 and 994 but not regions 993 and 994 (whichare TS coded regions), and look at U versions as shown in FIG. 17 . Step992 disperses the extracted data and the remainder data or dispersespartial versions of the document (those partial versions formatted andcontaining only data at or above the target security level (all TS leveldata (which includes TS, S, C and U data), or all S data (comprising S,C and U) or all C data and U)).

In step 979, the SC or sec-con level protocol determines whether singlewords are granularly classified (TS, S, etc.) or whether a line isclassified (context filter), or whether an entire paragraph isclassified (see FIG. 16 ). If a commercial/privacy filter is used toexclude all social security numbers, the organizational protocol is setat a granular level to exclude just social security numbers. Differentgroup protocols use algorithms to mark, filter and extract adjunctivesecurity sensitive words, characters, icons and data objects near thetarget SC or sec-con sensitive words, characters, icons and dataobjects. The SC or sec-con words may be security sensitive words,characters or data objects defined by compliance with law, regulation orpolicy, privacy, national, organizational or private security concerns.For example, “Bin Laden” is the target sensitive word in FIG. 16 andthis classifies the entire paragraph as TS level. The other words in theparagraph are adjunctive word/objects.

In a knowledge expander mode, the SC or sec-con filters are applied in anegative manner as follows: (a) in the user set-up, the user establishestaxonomic categories or classifications and sets the selection priorityof the classes; (b) the source document or source data stream is brokenapart to separate all metadata; (c) the source document (may be a datastream) is processed by a taxonomic filter which tags or labels eachword or data element with the taxonomic classification and thesensitivity or priority label (multiple “tax” tags, overlapping “tax”tags and “unknown” tags are possible); (d) after labeling, a content andcontextual filter is used to separate out the unknown words or dataelements (a negative filter). The resulting negative list of searchterms is used in various search engines for both public and private datasources, to compile a compilation of supplemental documents (suppledocs) and, thereafter, the supple docs are re-cycled through the H-taxand priority filter, then the content and contextual filter and asecondary supple doc collection is obtained. The primary and secondarysupple doc collection represents the expanded knowledge search nottypically found with commonly available search engines and processingtechniques.

C.15 DOCUMENT OBJECT MODEL (DOM)—PROTECTION AND PROCESSING

The search for expanding the knowledge base from a single sourcedocument to a larger compilation of docs has changed from locatingcontent (see the prior art GOOGLE search engine) to expanding the searchand search results for concept and context. Sequential text files arethe exception rather than the norm. Flat, plain, and sequential fileswould have disappeared entirely from all but transitional processingsteps except for the recent success of HTML web sites and the desire forstorage of complex data into sequential XML formats. In spite of theapparent linearity of HTML and XML, in practice these flat filesparticipate in a greater complex hierarchy of structured data mapped byobject models. The object models blur the lines between content,concept, and context such that effective security requires a broaderstroke than merely encapsulating content with encryption and limitingaccess with tokens or encrypted certificates.

Linkages to external files, style sheets, and embedded applications orscripts undermine the simplicity of HTML and XML flat formats andcompromise point security. Even structured field or line andrecord-oriented file formats have given way to more complex data storagemodels. It is insufficient to view security of content and files interms of encryption and encapsulation alone. Structured object modelsmix content with metadata and methods such that non-granular access—thatis, either/or barrier-based access through encryption keys, dongles, andpasswords—undermines any concept of effective security.

Furthermore, simplistic document management and access control overlookthe multiple purposes for each compound data document and the adverseimpact on organizational processes and work flows. Barrier-basedsecurity also fails from any Pacman-style attack, where the barrier,once breached not only provides full access to the once-protectedinterior also interferes with analysis of the attack and observation ofhow to prevent the ongoing attack. Granular multi-level control of userdata, metadata, data stored through the specifications of a hierarchicaldata object model, and methods underscores the new security paradigm.

All data sources important to data process workflow are non-linear,non-sequential, and not standalone in that the data sources areinterconnected to or required by other data sources. This includesdatabases, structured documents, desktop application user files,hierarchies of data structures, and work flows. The most advanced dataworkflow and the focus of attention is the object-oriented models usedin data processing today which comprise a cascade of events rather thana single point operation. This complicates SC or sec-con data expansionactivities to promote security, survivability, privacy, confidentiality,and anonymity. The present invention improves the security of complexdocument object models and interdependent workflow by expanding theknowledge base form a source document, thereby testing theclassifications levels and generally expanding the knowledge base of auser form the simple source doc.

There are only a handful of counterexamples to complex data structures,mostly monolithic file structures and simplistic processes. Thisincludes text files, raw binary image files, and lists. These aretypically inputs to older or uncomplicated computer activities; they donot reflect the complexity and interrelationships consistent with andnecessary for most critical networked data processing activities.Examples of flat files are text files, binary images, and lists.Plain-text documents are used only as temporarily or as conversion pathsfor other activities. Binary graphics are employed for their specificsimplicity, speed of display, and small size. It should be noted thatthey (BMP, GIF, and other formats represent the bulk of web images) areusually stored in an inverted backward last-to-first sequence. Listfiles are rarely important and standalone files are often a temporarypart of another process. One of the most ubiquitous of plain-text files,the HTML web page, is rarely a simple text file, but a circularconnection to many other like files and one part of a more complexhierarchy. A relative of lists is the field-oriented record structure.This is web page usually a grid-like storage of linear data. However,even a table grid, multi-dimensional indexing, SQL query concept isgiving way to object-oriented post-relational database storage methodsbased on object models in order to augment functionality, speed ofperformance, cross-platform and application functionality, and competewith easier to use user and developer products. Even the image files arebecoming increasingly complex. Hierarchical images formats with vectorgraphics compress motion and curves into small packages. Examplesinclude Corel Draw, Macromedia Flash, Adobe Photoshop, and MicrosoftPhoto. These of course contain proprietary andunintentionally-distributed information. Increased reliance on reliabledata storage infrastructure and networked storage technologies isenabling the transition to data storage based on object models.

FIG. 19 shows the root, branch, and leaf paradigm of this principal datastorage structure. See root 1012, content leaf 1014, branches 1016, 1018and leaf 1020. The object model refers to the layout or the map (ablueprint supplied by the document object model (DOM) vendor) of how thedata is potentially stored in what is definitely a linear file. Thestored file is the document object structure containing the data whereasthe model is the schema representation. The model FIG. 19 is just ablueprint for an empty data structure.

The data structure is stored as a binary file populated with datarepresenting a subset of that blueprint. The data file is often referredto as the document binary file so as to make clear that it is not aplain-text file, not in user-friendly format, and generally readable byan ASCII reader only in discontinuous chunks. The model and thestructure are not the same. The model (FIG. 19 ) does not represent asecurity threat in itself; it just represents how to find and definedata stored within an actual data structure. It is the data structure inmemory (the source document) or stored as a file that is the securitythreat. Usually, the file containing the data structure gives enoughclues to the purpose, methods, and sources unless addressed by amulti-level security scheme attuned to the complexity of the objectmodel. Although this “file” is stored as linear flat file, the extendedstructures is dependent on the hierarchical collection of potentiallyinfinite branch and leaf references. Despite this complexity, there areclear reasons based on simplicity for this hierarchical structure, notthe least of which is flexibility, self-documentation, andbackwards/forwards compatibility.

The subtle differences between a plain-text file, a file containinglightly structured data, the schema, and a file containing data withinan object structure becomes very important for security. When files aredistributed and those files each contain data within object structures,workflow is complex and cannot be effectively protected withbarrier-based security without complicating or disrupting operations.For these reasons, internalized security reflecting leaf content,structural paths, and the mesh of inter-relatedness among the paths,leaves, and external sources becomes the next paradigm for implementingeffective content-level and application-level security. Consider thedata structure defined by an object model as an organizing container.The contents within can be empty, or collections of containers, withmore containers within. It is a security sieve with traditionalencryption and the requisite inter-process work flows. The leaves andthe security of the leaves does not secure a chain of evidenceincreasingly necessary in modern data processing activity.

Enhanced security must reflect this distributed requirement since thedata sources are not single point sources, but complex relational,object-oriented, or hierarchical. In addition, data access andprocessing is approaching a worldwide distributed infrastructure, andcompletion transcends single places, times, and events. When thesecurity problem is dispersed, the security solution cannot bemonolithic either but must reflect the dispersed distribution andhierarchical complexity of the data and process. Location is not theproblem, so metaphorical perimeter walls are not the answer. To treatsecurity too as a monolithic, static, and walled solution when thesecurity problem is granular and dispersed within a flexible time framemisses its true need. Effective data security must reflect fiveinformational attributes in a newer paradigm for security. The fiveinformational attributes are listed below and examples of the attributesare also listed. For each select content SC or sec-con sensitiveorganization, the data structure must be analyzed and the fiveattributes must be applied to each root, branch and leaf to ascertainthe level of SC or sec-con sensitivity for that item. For example,applying a security schema to the SC knowledge expander problem, a TSlevel may be established by applying the five attributes that all audiofiles are “security safe” for that SC or sec-con level but these audiofiles will not be downgraded or released to a lower SC or sec-con level.Therefore, the meta data representing the audio file is designated TS.Another example is that all machines at the SC or sec-con level S(3^(rd) level of the TS-S-C-UC schema) are 2004 machines and programs.The organization may set, as a policy, that all MS Office program metadata need not be backward compatible beyond 2004. This organizationalprotocol then reduces security issues relative to the backwardcompatibility issue.

Informational Attributes Table for Security Purpose Sources and methodsOwnership Date or timeliness Content

Purpose Classification - Exemplary Table .backwards compatibility(purpose: communication across machine platforms .background color(purpose: visual presentation) .font size (purpose: visual presentation).image .video .audio .version control (purpose: source identification).etc.

Sources and Methods Classification - Exemplary Table .origin plain text.origin entire document .image .video .audio

Ownership Classification - Exemplary Table .source, author .securitylevel initial document .security level generating modifications toinitial document .hierarchical, orthogonal security classification

Date or Time lines - Exemplary Table .version control .sourceidentification (includes all contributing  authors supplyingmodifications)

These five security attributes reflect not only the data content butalso the point processes, embedded resources, and work flows. In asimilar manner, SC or sec-con sensitivity levels can be applied basedupon time (temporal) issues, competitor or size of company, type ofproduct (critical, sub-critical, or ancillary), etc.

This metaphor fractures complex data processing workflow. Traditionalsearch methods point only to defined search terms, maybe with a simpletruncating algorithm which deletes “s” and plural suffixes from thedefined search terms. A monolithic approach to prior art searchingignores metadata, process, multi-tiered files, delivery, and storage.Data sources are not monolithic, and certainly data is not either.Distributed data, distributed processing, and widespread distributiondefeats common search techniques. Access and search techniques need tobe granular and multi-level, and represent the five informationalattributes presented above.

C.16 RECOGNIZING THE DOCUMENT OBJECT MODEL (DOM)

Many word processed documents, such as those created by MS Office,binary documents which contain confidential information, typicallymetadata. This ranges—from small amounts of information aboutauthorship—to the editing history complete with deletions, reviewercomments, file attributes, and source and routing information—toextraneous baggage from documents previously edited during the samesession.

A multi-faceted SC or sec-con workflow process becomes an issue overcontrol of distribution by document type, recognition and categorizationof all user content defined by security exons (discussed later), removalof non-coding or non-activating security introns (discussed later),preparation and distribution by SC or sec-con sensitive levels, contentcertification and accreditation (C&A) subject to conversion to primitiveand certifiable file formats, distribution in print-representative-likepackages, with guarded ingress and egress of the source data files.Finally, implementation of a knowledge expander search throughgranularity of document node elements by analysis for inclusion andexclusion is a far more effective method, permitting collaborationwithin a multiple-usage infrastructure.

It is important to recognize that there are many file types and documentstructures associated with a document management program such as MSOffice. The structures are shared with other applets and competingproducts. Each program application, such as Word or Excel, create filebinaries or binary files with different object structures butinterchangeably read/write and import/export each other's file types,embed portions as formatted text or complete objects, or link throughremote procedure calls to these other file types. These object modelstructures are generically called the Document Object Model (DOM). TheDOM is another term for an object-oriented data storage package. Thepurpose for the DOM with hierarchical storage of metadata is three-fold.First, it is useful for backwards and forwards version compatibility.Second, metadata extends the document creation session from one-timeevent into an ongoing revisional process. Third, metadata provides orderand structure otherwise notoriously difficult for inherently free-formand flexible documents.

Metadata provides backwards and forwards version compatibility. Metadataprovides the necessary blueprint, format, and structure retention sodocuments can be revised in future editing sessions. Format informationis part of the DOM metadata, although style sheets and schemasmaintained in a different storage channel are valuable in HTML and XMLand might aid the future transition to a different DOM.

It is incorrect to assume a static basis for any document structure, asa monolithic DOS-based file, or as an in-memory object. For example, theExcel DOM can be embedded inside a Word DOM, which selectively can thenbe pasted as a formatted object into a PowerPoint presentation. Anotherconcern that arises in almost every document is imports, pastes, and OLEimbedding of other documents and aspects of the object modelcorresponding to that application type. For example, a base Worddocument with a spreadsheet and Project waterfall chart now includeseditable components referencing a different Office application with datain a structure referenced by that corresponding application objectmodel, in this case Word, Excel, and Project.

FIG. 19 shows each branch or leaf can be replicated indefinitely untilreaching the limits of RAM or file size. Each application has adifferent DOM. FIG. 20 shows the object model for word documents circa2004. Because of DOM evolution, with the assertion of backwards andforwards compatibility, realize that some nodes might exist in thebinary document file but not every function appears within eachpublished output because it is not used by the author.

A basic Word document file requires 18 KB on initial saving, but a full28 KB with edits and deletions, metadata, and redlining. Footnotes, fontchanges, hidden text, additional changes, headers, and footers, table ofcontent, indexing, an index, macros, .DLL add-ins, .OCX add-ins, andformulae could arbitrarily increase the file size indefinitely. Thisshows that MS Office security risks are reproducible at any userdesktop. A hex editor used in conjunction with an initial raw ASCII fileand the corresponding .DOC file also shows risks. ASCII text has only 40characters despite the directory display of the 1KB FAT32 block. Theinternal encoding of the .DOC file with initial content, the binaryobject structure and additional metadata are partially encoded in apadded form of ASCII. The metadata displays the source location of thedocument, removing possible doubts of file directory structures,security based on location obscurity, and other rational workflowtechniques for securing user files within the context of a networkinfrastructure.

Microsoft has identified thirteen key categories of metadata: Name;Initials; Organization name; Name of originating computer (desktop);Name of network server and/or hard drive; File properties and summaryinformation; Non-visible embedded documents; Names of previous authors;Document revisions; Document versions; Template; Hidden text; and Authorcomments. Some of this metadata is accessible through the Officeapplication menu interface through menus and dialog boxes. Theexploitation of this metadata in a knowledge expander search is useful.There are also the document file properties exposed by the Tools/Optionspulldown menu and the User Information tab. This is not the completelist of metadata. Other visible metadata with potential searchcharacteristics include: Footnotes; Cross-references; Table of Contentstags; Indexing tags; Hyperlinks; and Smart tags. Expect x-link andx-pointers plus style sheets and schemas within documents saved in theXML format. In addition, other undocumented structures are part of theextended and expanding Office document object models. Consider fieldsand mail-merge fields, which are markers for information automaticallyinserted by Office or by a user when opening, saving, printing, oremailing documents. These fields create a built-in facility for carelessinformation disclosure or overt hacking. There are also the documentfile properties exposed by the File/Properties pulldown menu. Thisincludes: File/properties; General; Summary; Statistics; Contents; andCustom.

The knowledge expander search also contemplates finding and using otheritems not specific to MS Office. Techniques for information camouflagecan be used in an knowledge expander search. These are equally valid inmost any desktop application, and are most relevant to presentationoutput rather than binary file delivery. Information camouflaged in asource document includes text set to small font sizes, such as 0 or 1,fonts set to type unlikely to be installed on the system which map tosymbols or line drawing, PostScript or Unicode font sets with alternateencoding, and font color set to match the paper color or an appliedbackground. White font on white paper hides text, black font on a blackborder or shading hides text too. Text can also be hidden with graphicswhen the graphics are anchored to a specific location congruent with thetext. Color games with text and graphics also hides the text. Macros,VBA (Visual Basic Application) codes, VBA add-ins, and applets alsorepresent a search asset. Anything than anyone can imagine as anapplication can run from within MS Office, productive or destructive.Usually, these bits of code are stored as part of the document metadata.However, they also can be out-of-channel files. Either way, they can becompromised by a new code that overwrites the original. They also can beinserted through fields, formulae, or menu add-ins. Collaborative toolsare the most obvious entrée.

New features in Windows and other Microsoft digital rights management(DRM) applications, such as ORAPI, ADSI, and MS IRM provide forcollaboration, resiliency, and complex versioning and backup far beyondthe capabilities of MS Office.

C.17 CONTENT

The differentiation of content within an MS Office document based oninitial owner and target distribution is important for search expansion.Some content will be strategic and some tactical. Content of MS Officedocuments transcends the actual presentation as a printed page, slide,spreadsheet, database report, email message, an index of documents, UML:or project waterfall, or organization chart. Microsoft Corporation ispositioning Office as a platform for delivery of new services; it is notjust about a PowerPoint presentation or a Word document printed to afacsimile. The DOM is a project plan, with a structure, with componentsthat do things and are sensitive of their own.

For these reasons, it is important to explore the MS Office DOM factors:Content classification; Tagging; Clearance level; Data mining; Trafficanalysis; Inference; Encryption; Digital Signature; Document accesslinked to Fortezza (an encryption program/system), PC Crypto cards,smartcards, and n-factor authentication; Granularity; Strategicinformation; Tactical information; Common Criteria or NIST analysis;Covert channels; and Bell-LaPadula model conformance.

Content classification with taxonomic classes occurs with tagging forformatting with bold, indexing, and paragraph marking, explicit elementtagging for HTML and XML or database and spreadsheet table, field,ranges, row, and column designations, as well as authorship techniques.Formulae and macros define ranges with informational content (contextualalgorithms which link content), as well as indicate purpose and intentof the process as well as the target data. When content is tagged at thesideline, as in “eyes-only,” or within-the text with any label name forclearance level, as in “<1>,” this attests to a SC sensitivity levelwith an importance factor. For example, a subtotal of employee salarieswithin a pro forma business plan matched against a list of employeenames compared to a bank check ledger gives away each employee's salarylevel; each document in isolation does not give away information untilseveral are merged and analyzed together. Direct analysis through recordrelationships and sorting is one type of data mining, human intelligencethrough inference or statistical inference with set theory or Bayesianmethods is yet another. For example, because you know that 6 employeesare traveling to a conference in D.C. and two others are not in theoffice, you can approach a particular person who by inference is manningthe station desk with a very specific social engineering attack.OneNote, InfoShare, Net Meeting and/or Live Meeting, Outlook, andExchange with MS Project also enable workflow routing, group editing,and acceptance sign off. This information becomes part of the documentmetadata so that traffic analysis shows where the document originated,what changes were made and by whom, how it was routed by username,network, and IP address, who has seen it and has access to it, and allprocess flow and comments. One of the secure prizes of organizationinformation thus unintentionally published is the names of people withinthe organization and functional roles.

Designing a knowledge expander search engine through granularity of MSOffice node elements by analysis for inclusion and exclusion is aneffective method. Multiple source documents create structure andsemiotic meaning not in evidence with subsets. This process breaks thecontext to prevent useful data mining, routing inferences, and the morepowerful semiotic information methods. It allows for the separation ofstrategic information from the tactical, so that access is granular byrole, user, and other discriminators. Many academic and implementedsecurity models are in use today, both as a straw man and forcertification processes.

C.18 DOM PROCESS EDITOR

Document object model (DOM) source documents, and particularly Officedocument modules, comprise the blueprints, process, external datasources and linkages, and materials for building the resultingpresentation; the presentation content is usually the ultimate endproduct. The blueprints and process often are immaterial to thepresentation and represent proprietary and confidential material. ThisDOM object model complexity and diverse accessibility creates anopportunity for the knowledge expander search engine.

Effective DOM and metadata searches use the object hierarchy structureas variously described as a binary tree, category structure, or hive. Inany event, the entry point is the root or base, containing a potentiallyinfinite number of subcategories, each with a potentially infinitenumber of leaf items. See FIG. 19 . The structure can be pruned,deleted, or rearranged. The items represent object-oriented information,from entire subdocuments, to relational databases, layered graphics withvector elements, to simple plain-text, to a single binary numericalelement.

The process requires a parse of all branches to each and every leaf.This process is not recursive, just extensive. Each path is examined forcontext, each leaf for content, all nodes for external references, andeverything must be viewed within the context of sources and methods, notjust obvious content. The obvious content is what the user created andsees, but as you now know, that is a minor portion of the data containedwithin the document object structure. This is a paradigm shift is shownin the hierarchy below:

DOM Deconstruct Table For each document (the file and structure);    Access the root;      For each limb;     For each branch;   For eachsub-branch;  For each leaf (item); and    Process each leaf.

Preservation of the path to each leaf is important as it defines theaccess to that data element. The existence and/or null value of the leafrepresents a security control point. The model defines, withsupplemental external knowledge of the object model, potential searchopportunities. The model and the content are not separate from externalknowledge of sources and methods. The model and the content are part ofa contextual analysis of the source document or data stream itself. Theleaf the significant search control point. It is possible to review andalter the contents of the leaf within the context of the purpose of theleaf to retain functional access with multi-level SC or sec-consensitivity.

C.19 FIVE INFORMATIONAL ATTRIBUTES

The five information attributes of SC or sec-con sensitivity in contextto processing include the leaf, purpose, sources and methods, ownership,date or timeliness, and content. The entity must establish protocolswhich rate or prioritize the five information attributes on each root,branch and leaf in the DOM source document. With the system initializedin this manner, the processing of the DOM document within the parametersof the knowledge expander search is accomplished.

Purpose: How does the purpose of the leaf provide context, purpose, orinformational reference to the document as a whole or the individualleaf? Does it provide source, destination, authorship, viability,validity, verification, or integrity to the document as a whole or theindividual leaf? Consider the value of processes imbedded in thedocument as cell formulae, a help file, or other complex routing wizard.Does it show linkages or references to other documents? What is itsstatus or position within the document? What is its element position, asa headline, footnote, or redlined status? These seemingly minor detailstranscend actual content but provide clues to the following attributes.

Sources and Method: Intelligence agencies stress the confidentially ofthe sources and methods used to gather information. The informationitself might or might not be important, but the ongoing care of thesources and methods is important for future information gatheringactivities and retention of any status quo until action is initiated. Inaddition, the viability, validity, verification, or integrity of thedocument is predicated by the viability, validity, verification, orintegrity of the sources and methods used to create it. In terms of theOffice document, this type of information is both contextual, leafcontent, and metadata. To presume that the SC search uses only doccontent at the leaf misses the value of metadata and the inherent valueof the object-oriented document format. For example, authorship, source,source dates, editing dates, deletions, redlining, notes, footnotes, MShidden text, links, and other structural elements in the source docdescribe when, how, where, and who created the document. This speaks tothe viability, validity, verification, or integrity of the document as awhole, and can compromise past, ongoing, or future data collectionefforts and operations.

Ownership: Ownership is reflected both in leaf-level content—that isobvious when a document is presented or published—but also in themetadata. Ownership is also a characteristic of file storage properties,in ring rights, file storage position, linkages, SMB or network fileaccess rights, and HTML references. Ownership, particular the number oflinks, the times access and edited, numbers of hits, and the level ofchurning, suggests the relative importance and merit in the document.

Date-Timeliness: Date or timeliness reflects currency. The dates, interms of edit times, access times, and frequencies suggest the relativeimportance and merit in the document. Touch and other file-levelcommands can only mask the overt date and timestamp of a file, not itspurpose or content, true timeliness, or merit. This information isspread through the metadata and leaf content. In some hierarchicalstructures, this information is stored in tables or other structuresapart from the immediate document root. When a document is a relationaldata structure, as in Access or SQL, hidden system fields and hiddensecurity data define edit and deletion times. It is also important torecognize that in databases, records which are deleted by the user areonly marked as deleted but persist until the database is purged, packed,cleaned, compressed, or otherwise processed in a maintenance mode. Whenrelational technology with transactional logs and rollback facilitiesare enabled, data can be recreated or dated despite many types ornatural of instigated disasters. This supplemental metadata defines dateand timeliness too.

Security: Security of content can be compared to erecting a barrieraround that content and may be viewed as an important SC sensitivityissue. However, when content becomes a collection of simple dataelements along with data objects, dispersed and distributed sources,effected by embedded events and triggered methods, a search whichignores the security content characteristic fails to acknowledge thatthe SC sensitivity may be impacted by such omission.

While content is king in most search systems, it is not the onlycritical aspect of a source document. In terms of the knowledge expanderprocessing of an Office document, each leaf must be processed andassessed for its metadata. Note again that each leaf may be anotherobject-oriented structure in its own right or a simple element. It willneed to be processed and assessed accordingly.

C.20 SELECT CONTENT INTRONS AND EXONS

Terminology employed in connection with the operation DNA(deoxyribonucleic acid) provides an appropriate metaphor for the MSOffice document object model or any other DOM model. While the DOM isseparate from an MS Office binary file, it defines the purpose of thatfile and maps its activation. The DOM “genes” are expressed into thefile binaries only as specifically referenced, and frequently divergefrom the pure MS Office application as genes from other OLE (objectlinking and embedding) applications are embedded into the document. TheDOM and the expressed document can mutate for better or worse, and bothbackwards and forwards the document is adaptable just like DNA, withunforeseen consequences including the profound security flaws evidentwithin the MS Office workflow.

In genetics, an intron is any non-coding or non-activating sequence ofDNA initially copied into RNA but cut from the final RNA transcript orunknown as to singular or recombinant purposes. Introns are excluded orignored in the DNA process. An exon is a coding or activating sequencewith a known purpose that is actually used or one that is unknown as topurpose but nonetheless still used. DNA is, of course, the blueprint forlife. RNA is the functional transcript of the DNA blueprint used forcell division and replication. Exons are the useful portions in the DNAcycle.

In the object model, the DOM is metaphorically the DNA blueprint for anMS Office document whereas the actual Word, Excel, or Outlook message isan expression of the RNA as a functional transcript. Correspondingly,the SC select content or sec-con intron is any document branch, leaf, ornode element with a non-coding, non-activated, or even unknown controlutility for the document. From a select content-search (orsec-con-search) standpoint, each and every intron represents anon-qualified element that is a potential search term that may lead toan expanded knowledge expander supple doc compilation. Rather thanignoring SC or sec-con introns as most common search engines do, thecurrent knowledge expander search engine employs the SC or sec-conintrons to expand the scope of the search. A SC or sec-con exon is anydocument branch, leaf, or node element serving an end purpose. Each SCor sec-con exon in a MS Office document becomes a certifiable dataelement.

Unless each such SC or sec-con intron and SC or sec-con exon in thesource document DOM is vetted for credentials, which vetting includes aprioritized taxonomic label or tag, the knowledge expander search,whether through conventional search engines or the novel search enginesdescribed herein, will be incomplete. This is effective for DOM, HTML.XML, databases, and any structured file binaries. The standard 2-phraseprocess transforms into a 3-phase process where DOM node elements arecoded either as exons or introns, and thereafter processed accordinglyfor inclusion or exclusion.

The improved accuracy of 3-phase scanning of documents within thecontext of an object model is apparent. While the traditional 2-phasemethod find SC or sec-con introns and SC or sec-con extrons within asource document, it also miscodes several other sequences as introns.The accuracy of such process will always include statisticallymeasurable false negatives and positives, thereby missing true threatsand removing non-threats. The 3-phase process improved on the 2-phaseprocess with granular deconstruction of the document and subsequentrecoding of both false positives and false negatives to yield a higherrate of accuracy. SC or sec-con introns are added to the search termlist in the knowledge expander search engine and are scalablenumerically. Better DOM maps mean better intron handling.

FIG. 21 shows the General DOM Editor program 1022 in a flow chart form.Step 1024 is the initialization that is employed by the security entityor SC or sec-con sensitivity supervisor to set up the program. Step 1026obtains the DOM layout or blueprint from the DOM vendor. Step 1028 notesthat for each root, branch and leaf in the DOM, the informationattributes must be prioritized with the SC select content or sec-con andorganizational informational attributes (a taxonomic routine withprioritization). SC or sec-con introns are identified and SC or sec-conexons are classified, placed in a hierarchical structure and labeled.This step, of course, uses a content filter to identify the SC orsec-con exons. A contextual filter or algorithm is used to taxonomicallyclassify SC or sec-con exons at a higher priority level. For example,when “Bin” is next to “Laden” the SC or sec-con exon combination—BinLaden—is classified TS or the top level for the SC or sec-con H-taxclass.

Step 1030 obtains the source document DOM. Step 1032 maps the DOM sourcedocument as a binary file populated with content data and meta data as asubset of the DOM blueprint. Step 1034 identifies SC or sec-con exonsand SC or sec-con Introns. Step 1036 labels or tags SC or sec-con exonsper the tax class (taxonomic classification) and per priority, to beincluded in the further processing of the item and identifies SC orsec-con introns to be used as search terms in the knowledge expandersearch. Multiple tags and overlapping tags are possible and aruled-based system is employed to upcode multiple tags and overlappingtags to the next higher tax class—priority level. Step 1036 converts SCor sec-con exons at each root, branch and leaf into SC or sec-con levelsave doc, form and format (for example, a safe DOM template), andgenerates a save blueprint or map, drops (or alternatively stores in aremainder store) all low priority SC or sec-con exons, and expands thesearch term list with the SC or sec-con introns and the top group of theSC exons. Step 1038 processes the save DOM doc, that is, the top levelSC or sec-con exons and all SC or sec-con introns as search termsthrough public and/or private databases, indices, search engines, etc. Adivergent search uses SC or sec-con intron. A convergent search uses SCor sec-con exon. Supplemental documents are gathered form the search,the filter H-tax class priority tag process is repeated on the suppledocs, the SC or sec-con introns are identified and selected top prioritySC or sec-con exons are identified and the search and a second tiersupple docs compilation is processed. Supple docs are stored and arelational map is generated both with the search terms and the suppledocs.

FIG. 22 shows a flow chart of a basic application for a DOMEditor-knowledge expander search for MS Office document. Step 1042initializes the system. Step 1044 obtains the source DOM document andnotes that all processing occurs after a spam and virus filter. Step1046 notes that the program creates or is provided with a security safeor clean DOM document and map. All levels of SC are labeled or taggedand saved in the “save doc” routine. This hierarchical taxonomic (H-tax)tagging labels all SC or sec-con words, terms, etc. and locates and mapsexons and introns in the source doc. Step 1048 notes that a template isopened and SC or sec-con exons and SC or sec-con introns (non-standardSC or sec-con and unknown terms, characters, words, etc.) are copiedfrom the source doc into the clean DOC. A clean .DOC template (Word) orwhatever the new document type is opened for the specific application.The Normal.DOC or New Spreadsheet.XLS on the MS Office distribution CDis safe. In Visio, for example, start with a new chart. Older documentssaved as templates tend to accumulate unanticipated metadata. If onemust use a non-standard template, clean it up. View it as both a printeddocument, as a binary file, and as a document object. Write protect thetemplates, or store the templates as non-modifiable networked volume. Ifthe templates are used throughout the organization, create a networkstore for them and configure each user's installation of MS Office tolook there for the templates. Search the network for duplicate copies ofthese templates and eliminate them.

If changes from any version of MS Office to another version aremade—this can be a regularly upgrade or a even a downgrade—create newdocuments and cut-and-paste parts of prior documents into new ones. Losethe older files and templates. If you receive or open an Office documentin either an older or newer version, create new documents andcut-and-paste parts of prior documents into new ones consistent with theMS Office version that you use.

Step 1050 disables edit controls and step 1052 copies SC or sec-con exonand SC or sec-con intron content. The point is one must edit cleanly.This is not a single step but rather a process, both one time andongoing. Disable versioning in step 1050 to prevent a buildup of pastversions of the document. With versioning, prior sessions will be storedas document.doc 1, document.doc 2, and so on. These tend to get entwinedwith the latest version. If workflow with InfoPath, Outlook, or othercollaborative workflow tools creates duplicate copies of the sourcedocument file for each user. Step 1050 includes the concept that thesystem is configured to store a single network copy instead. Preserve anaudit trail and backup with a system backup rather than versioning.Disable change tracking in step 1050 to curtail the buildup ofadditions, deletions, and changes that transcend the publishing intentof the document. If redlining is necessary, establish guidelines forperiodically accepting changes to purge change log. Use the command tosave the document without the change log with File/Save or File/Save As.Do not use nor rely on fast saves, timed saves, or file recovery after aMS Office crash to purge the dirty metadata. After a crash, reopen thedocument, save the document under a new name. Close the Officeapplication. Delete the old file precipitating the crash. Rename the newfile under the old name. Reopen the Office application.

Step 1054 locates text in footnotes, hidden text, etc and labels or tagsthat content as SC or sec-con exons or SC or sec-con introns and mapsthose data elements to the SC or sec-con save doc and copies theelements into the Clean DOC. Use comments instead of hidden text. It isdocumented as a feature so it can be found rather than accidentallyuncovered. Hidden text with a font color change or font size changelooks like an artifact that most users will ignore or delete. Avoid theuse of headers, footers, footnotes, endnotes, inserts for table ofcontents, index and the like. These appear only in the printed outputunless specifically viewed from the View pulldown menu. Such links alsocreate a lot of dirty metadata beyond what is visible even duringediting that persists until changes are accepted. Remove references fromthe source document. This is subtle, but very important when documentsare specifically posted or even inadvertently listed on web sites.References include other files, documents, hyperlinks, and otherpossible embedded formatted materials. These references create theability to infer quite a lot about the purpose of the document fromother related documents in the same directory, by the same authors, andthe types of other documents. For example, a spreadsheet stored with areport that is not overtly included in the report suggests that issource material that has not been reviewed with an eye towards privacy,security, or client privilege.

Paste and copy images, cut text, formatted text, pie charts, recordsets, slides, waterfalls, milestones, organizational charts as plaintext or an image rather than formatted Office objects. If the embedcommend is used, all the metadata baggage from the other Officeapplication is now added to the metadata in the target document. Sincethat metadata baggage is not native to the target document application,it is inaccessible and truly hidden. Tools, such as Metadata Assistantwill not find Excel metadata within a Word Document, Word metadatawithin an Excel spreadsheet, and none of them within an Outlook note ormessage.

Step 1056 notes that a clean map for the meta data cleared DOM documentmust be created.

Step 1058 executes the intron search and/or granular search routine,gathers and compiles supple. docs and the processes those supple docsthrough the granular filters discussed earlier to locate, with thehierarchical taxonomic filter process tags new words, terms, symbols,which are related to the original SC select content or sec-con, therebyexpanding the knowledge base of the meaning of the SC or sec-con sourcedocument.

C.21 HIERARCHICAL TAXONOMIC CLASS EXAMPLES

Various type of classification systems (taxonomic systems) may beemployed. For example, a dictionary classifies all words as nouns,verbs, adverbs, etc. This is one taxonomic system. A prioritized H-taxsystem classifies nouns into a name subclass and this name subclass maybe priority or sensitivity level 1. Geographic locations (class nouns)may be priority 2. The operator of the present knowledge expander searchprocess may place a high priority or sensitivity on “noun” class,thereby converting the simple taxonomic structure into a hierarchicaltaxonomic system. Identifying categories and subcategories for SC orsec-con sensitive words, etc. or critical data, creating a compilationof pre-existing data, comparing the compiled pre-existing data to thetarget data and labeling or tagging the SC or sec-con terms is one ofmany important aspects of the present invention. Table 1 in FIG. 23provides categorical identifiers for personal identity. Thesecategories, which may be utilized to identify a person, are not meant tobe all encompassing but are mainly provided as examples.

After categories are identified for the critical data, it is importantto create the hierarchical taxonomic system against which the targetdata is tested. Ranking or qualifying the categories at SC or sec-consensitivity levels is next. Table 2 in FIG. 24 is a general attempt toquantify the categories for personal identification from a high riskvalue beginning with “name” to a low risk value ending with “personalinterests”. Again, the Ranked Identity Category Table 2 is not meant tobe limiting but is meant to be an example. Individuals skilled inidentifying a person may alter the ranking of the identity categories inTable 2, FIG. 24 .

The present invention can be employed to use a hierarchical taxonomicsystem for a business. Table 3 in FIG. 25 set forth below providesexamples of categorical identifiers for a manufacturing business. Again,this list is not meant to be exhaustive or complete, but is onlyprovided as an example of the types of categories and subcategorieswhich a manufacturing business would employ in order to establish therisk monitor of the present invention.

With this list, the manufacturing business may assign a SC or sec-consensitivity level to each category (class, subclass and sub-subclass)thereby creating a prioritized hierarchical taxonomic system.

C.22 KNOWLEDGE EXPANDER (KE) BASIC PROGRAM

As discussed earlier, the information life cycle or knowledge expandersearch module can be configured with many different modules in order toachieve the objective of the operator. The knowledge expander or KEbasic program 3001 in FIG. 26 is one basic implementation of theexpander search program. FIG. 26 , In step 3002, the operator or usersets various filters and sets the search targets. The search targets maybe publicly available search engines or private data bases, datacollections, indices or any item that the user wants the KE search toaccess. Filters are described earlier as hierarchical taxonomic filtersor taggers, content, contextual and other types of filters. Filters areused to identify class, subclass and priority in the hierarchicaltaxonomic or H-tax system, to apply contextual rules or algorithms(“bomb” within 10 words of “arab”), and content (select content SC orsec-con verses non-matching SC or sec-con). Step 3004 obtains the sourcedata or the source is input by the operator. Data input may be from aclient computer in a server-client computer system. The source data maybe a data stream, a source document or other item. Step 3004contemplates that if a document is utilized, the document isdeconstructed to its basic data elements and appropriately mapped. Seethe discussion above in connection with DOM branch, leaf and treedeconstruction. In step 3006, the hierarchical taxonomic classification(H-tax) occurs and each data element in the input document is taggedbased upon the classification system which has been prioritized by theuser (or the user uses a pre-set prioritized H-tax). A note todeconstruct the source document into data elements (see DOMdeconstruction into its object hierarchical structure above) is found instep 3006. In step 3008, for non-matching data elements, a search isconducted through the search targets. Non-matching data elements formthe input doc are those which do not match SC or sec-con. In step 3010,the system gathers the documents from the search (supplementaldocuments) and repeats the H-tax tagging operation for b number ofsupple docs. A relationship mapping function and display function isactivated if c number of hits occur within the H-tax top priority range.In other words, if the search for non-matching data elements returns 50select content or SC or sec-con terms and of those 50 SC or sec-conterms in the supple docs, 20 SC or sec-con terms fall within priorityranges 1-5 of the hierarchical taxonomic classification set by the user(n priority H-tax levels), then the system and process maps the resultsand displays the representations of the H-tax recovered from the suppledocs c is less than 20. If less than b number of H-tax tags are noted inthe supplemental documents, the system repeats steps 3006 and 3008 andgathers a second tier of supplemental documents.

Steps 3012, 3014, 3016 are optional. As an option to be set by the useror as an option to be set by the system operator as a default, step 3012changes the H-tax top range bandwidth from n priorities to be searchedto m priorities. This change may be needed to expand the searchbandwidth from b to b−10 to b+20. A reduction (m less than n) is neededif too many supple docs are recovered by the search. If too few arerecovered, then m is made greater than n priorities in the H-tax. Option3014 changes the output supplemental document bandwidth to b+y if toofew supple docs are recovered. Of course, the output supplementalbandwidth could be reduced with b−y. Optional step 3016 mixes n numberof select content search terms with the no-match data element before thesearch. This effectively is a convergent filter. By adding the known SCor sec-con to the search terms (initially, search terms are generatedfrom the input doc based upon priority H-tax), the search is biased oris set to converge on the known SC or sec-con added to the search termgroup. By using the no-match (NOT AND) function, the KE search divergesto gather a wider range of docs. Of course, the user may always employcommon search engines in addition to the search plans set forth herein.Step 3018 repeats the process until b supplement documents or c hits (SCor sec-con sensitivity matches) are noted within the H-tax top range.Step 3020 maps, displays, stores and ends the program.

FIG. 27 shows an expansive or expansion unknown search engine 3031. Step3032 initializes the program with preferably user input, sets up thehierarchical taxonomic H-tax classification system along with a priorityor sensitivity level assigned to each level and branch and leaf in theH-tax system. Select content SC or sec-con filter modules for contentand contextual range or algorithm tests are also set. At least someselect content should fall within the prioritized hierarchical taxonomicsystem. The select content SC or sec-con encompasses words, terms anddata objects that the user wants to expand his or her knowledge basewith the search engine. The user sets a no-match search bandwidth nmsand sets search targets such as search engines, public or privatedatabases, data compilations, indices, data collections and datastructures or whatever. Step 3034 obtains a source or data stream orinput document. Step 3036 processes each data element (which may includea document deconstruction) with the H-tax, content and context filters,and tags each data element with an H-tax marker. Multiple, singular, anda default “unknown” H-tax classification indicators are linked or taggedon every data element. The system in step 3036 may upgrade or downgradethe priority or sensitivity H-tax level for a particular word, term,etc., due to contextual rules in the context filter. In step 3038, thesystem conducts a search for nms number of not matching or unknown dataelements. If nms is 10, the system takes the top 10 priority H-taxsupple SC or sec-con terms obtained from the input doc and uses the top10 supple SC or sec-con as search terms in the target data collections.The search is conducted through search targets identified in the set-upstep 3032. The no-match search 3038 is sometimes identified as step Aherein. Step 3040 gathers supplemental documents. Sometimes, step 3040is step B herein. In step 3042, the system processes the supplementaldocuments through the H-tax tagger. The supple docs are classified withthe H-tax. This is sometimes step C. In step 3044, the system determinesif p hits have been noted in the H-tax top range levels. If YES, thesystem stops and maps the select content SC or sec-con sensitivity sel.levels to the H-tax mapping structure and displays that SC or sec-conhierarchical representation map to the user. Further, the system in step3044 maps the supplemental documents to the H-tax map structure anddisplays that to the user. Sometimes, step 3044 is step D herein. Instep 3046, if p number of hits are NOT identified in step 3044, steps A,B, C, D are again executed and repeated unless the repeat cycle is morethan R. The nms bandwidth may automatically change (increase) or the npriority H-tax may change to m H-tax levels to increase the supple docsor the hit count for supple SC or sec-con. The same techniques may beused in reverse if too many supple docs are recovered. The system endsafter step 3046.

FIG. 28 shows knowledge expander KE search options 3051. Step 3052 is anoption to add a noise filter after the H-tax tagger step 3036 in theexpansive unknown search engine routine 3031. The noise filter reducesthe number of unknown data element and operates on the unknown tags forthose unknown data elements. For example, terms “and” and “or” and “the”may be deleted by the noise filter. Optional step 3054 adds to step D,if p hits is equal to zero, then the system resets the no-matchbandwidth nms+u and repeats step A, B, C, D and repeats the entireprocess until p hits in step D or R iterative cycles have been noted(counted). In optional step 3056, a convergence filter is added to theexpansive, unknown search engines 3031 by adding w select content SC orsec-con search terms to the no-match search step A. In optional step3058, a divergence filter is added by changing, either increasing ordecreasing, nms value if p hits do not exceed p+x. In optional step3059, a content filter is added to detect temporal relationships such astime and date ranges. Time may be time of length of search or time maybe a time-date range limit on recovered supple docs. The temporalcontextual filter (time and date is a matter of context) would be addedto step 3036 the H-tax step. In optional step 3060, the H-tax map isoutput to the user so the user sees the classification and theprioritization of that classification. In step 3062 the operator canchange the repeat cycle R, the time the search runs, the number ofcycles, and the search target databases, data sets, spreadsheets orpublic or private search engines. In optional step 3064, the operatorcan limit the supplemental documents based upon a temporal factor suchas time, the total number of SC or sec-con terms located in thesupplemental documents, and the total number of supplemental documents.

C.23 SECURED METASEARCH ENGINE BASED ON GRANULAR DATA CONTROL

As an overview, the secured metasearch engine technology is based ongranular data control. The engine is part of the KE engine describedabove in sections C.9-C.22. The secured metasearch engine technologyenables secured searches and distribution of secured search resultsincluding secured information sharing with multi level security (MLS)search capabilities across domains. Data is explored not only by contentbut also context and concept. Search should occur across domains, withinformation shared and delivered across domains and through theextraction and integration of both private and public data sourcesincluding the Internet. Stovepipes and silos, independent efforts atinformation collection and intelligence analysis, and air gaps orsecurity guards divide information domains and prevent securedinformation sharing. Centralization and consolidation of data gives riseto data overload, a problem in part addressed by new web-like searchingtechniques, but also opens the door to security leaks.

The security of current data mining and search operations as well as thesecurity of the multi level distribution of the search results to userswith different access levels is questionable. Furthermore, the currentdata mining and search is rigid because the starting point for each andevery effort requires an index or search terms and the mining iscompletely content-based. When connecting the dots, it is essential thatsearch be possible without preconceptions of utility, purpose, orcultural bias. Effective search techniques, called “metasearch” herein,comprises more than web-like textual (and other formats such as audio orvideo, but also compound meta-formats) search technology. It includesdata mining, textual and contextual search, aggregation, inference, anddata-to-data interaction all based on granular data control subject tosecurity compliance.

Metasearch generally means the usage of multiple web-based searchengines based on keywords with results combined into a single unifiedreport; this is better described as a confederated search or federatedsearch. Each web-based search engine is rigid in terms of its relianceon preexisting indexes; it is usually fast but limited in terms of mostlikely usage bias. However, the combined results reinforce thepreexisting answers based on the implicit search question.

In contrast, the Secured Metasearch Engine (sections C.9-C.22) isdesigned for searching the unknowns and expands the base of information,as well as conducts the search operations across different domains withdifferent security levels. The Secured Metasearch Engine resultsprovides two distinct and fundamental directions, search convergence andsearch divergence, both of which are critical to the process ofconnecting-the-dots. Failure to consider both convergence and divergenceleads to incomplete results.

Search divergence is the process of expanding the content, context, andconcept beyond the strict scope of the fundamental search. Searchconvergence is the process of validating, substantiating, andestablishing the integrity of the fundamental search. Divergence is theexpansion of the search query or data mining operation. There is a needto find outliers useful to expand the scope of the users knowledge andinfer new information and intelligence. Some information andintelligence operations require exploration without preconception.Herein, outliers with the least priority, relevance, or rankingfrequently provide previously unknown references or interconnectionsthat are purposefully hidden. This is a search divergence because thebase search query primes the process but effectively we already know thehighest ranking results and reject those results while iterating thesearch for non-ranking items.

The Security Metasearch Engine starts with known keywords and indexes,but accepts primarily information that is new or different from thedirect response to the implicit question, that is the base search query.Validation and integrity is not important to divergence. The searchengine is looking for all related metadata, references, links, andrelationships without regard to accuracy. Exact matches are in effectusually ignored, but all new references are added into a secondary queryyielding subsequent results (supple docs and supple search terms). Thesubsequent results in turn yield yet more new references. Thissubsequent query yields subsequent results, which is repeatedindefinitely. The iterative results create an asymptotic adjacency listmodel, which may include a networking relatedness. The networkingrelatedness is often viewed as flow charts showing betweenness,closeness, and connectedness. The networking relatedness may also be asocial network relatedness.

This expansion of the base search query is the process by which the userdiscovers more dots. Connecting dots the user already knows is pointlessand does not expand the user's knowledge; the user wants to expand theset of dots and generate new content, new context, and new concepts. TheSecured Metasearch Engine starts with old and/or limited information,but generates new information. The theory of six degrees of separationhas some legitimacy, however there is a difference between accidentaland active relatedness.

Divergence is also critical to enabling the Secured Metasearch Engine tofoster security by exploring the purpose and usage of information beyondthe immediate and obvious preconceptions (i.e. foresee reflexivity) ofsemantic and pragmatic meaning. See beyond the immediate securityperception for other possible cross-domain requirements and enableinformation sharing. The Secure Metasearch Engines is connecting knowndots with new dots, the true concept of “connecting the dots”. The newdots lack integrity and substantiation. They are not ignored ordismissed outliers because they can reveal the outlines of previouslyunknown, undiscovered, unforeseen, uninvestigated and unrevealedsituations.

The search process is really a metasearch, applying techniques of datamining, textual and contextual search, aggregation, inference, anddata-to-data interaction, and the integration is the establishment ofrelationships. However, some of the new dots will be false and noise. Atthis point, a search convergence is useful to establish likelihood andrisk for the new information and integrity and substantiation of thedifference between accidental and active relatedness.

The Convergence Search is described herein. Convergence is the usualweb-like search engine or data mining operation. It is rigid. The userstarts with known keywords and indexes—search terms—as the base searchquery and accept hits with the highest relevancy. This information isused aggregately to confirm or deny the integrity of the initial search.The user gets back what the user expects with priority, relevance, orranking focusing the hits to the content most consistent to the searchterm context. Outliers are naturally rejected and filtered out. Searchconvergence might entail a physical inventory, database reportverification, an independent party confirmation, or a drone flyover tosubstantiate the information. The result is in direct response to theimplicit question, that is the base search query. The user positions andsubstantiates the dots the user already knows. The user starts with oldinformation, and the user gets back old information. However, thebenefit provides by the Secured Metasearch Engine is that itcontextualizes these convergent results with a higher level of relevancyto the user's specific purposes based on tools which manage and controlgranular data. Those include tools include: (1) metafilters forautomatic categorization; and (2) sensitivity level taxonomy. Thesetools deliver “fine adjustments” to the search process and find the mostappropriate results for the user prioritized according to the highestrelevancy.

The combination of divergence and convergence operations providesadditional information. The Secure Metasearch Engines performs three keyfunctions. (1) First, it selects new data elements of value andrecognizes unknowns for automated iterative divergent search. (2)Second, the combined divergence search delivers “fine adjustments” tothe divergence search process and finds the most appropriate results forthe user prioritized according to the highest relevancy. (3) The SecureMeta search Engine categorizes all aggregated results to sensitivitylevel and enables distribution subject to MLS-compliance cross domain.While the Secure Metasearch Engines primarily focus on unstructuredinformation, it can be integrated with structured data sources toobecause it is actually easier to extract context and concept fromstructured data.

There are multiple starting points to the metasearch process. One ormore candidate data streams, news feeds, or documents can be draggedfrom the desktop into a queue. These items may be monitored for changesand then fed into the Secure Metasearch Engine for granular dataselection and metatag categorization with the output result that searchterms are generated and metasearch terms are also generated. A documentor data stream automatically sets the context of the search and isexplored for likely search terms/key words. No index or bias isnecessary to begin the process. The resulting search terms of interestare fed into the next search cycle, which may include also one or moreURLs for further exploration. Indexes and search terms do not define theimplicit question or establish parameters for an effective answerbecause they represent a single dimension of information. Metafiltersare used to control the granular data elements in order to fine-tune thesearch. The metafilters add both context and concepts to the initialsearch and all subsequent exploration. Such metafilters may be: (1)Categories of the granular data elements in a document (“name”“location” “date” “amount”); (2) sensitivity levels of the data elements(“top Secret” “Private” “level 9”); (3) phrases, (“Mohammed Atta”“United Nations”); and (4) Group Lists, (Groups of words which representa concept or a context—“Nuclear”+“Submarine”).

Generation of keywords/search terms for conducting the continuingsearches is done by controlling of data at the granular data level.Control of data at the granular data level is enabled by combiningdifferent filters. The Secure Metasearch Engine recognizes complexcontent and context by implementing a combination of filters/sieves.Those combinations of different filters/sieves create differentpermutations of resulting keywords for the search. The differentkeywords for the continuing searches are selected automatically basedon: (1) the words entered into each filter/metafilter; and (2) theconfiguration of the different filters/metafilters combinations. As anexample in one embodiment, all the content of a document is: (1)categorized, i.e. each individual word (or phrase) in the document istagged by category, “name” “location” “date” etc.; (2) the individualwords are also tagged by sensitivity tags “secret,” “confidential”“Private” “level 3 sensitivity” etc.; and (3) all the above is presentedin a matrix format. The filters will automatically select which words inthe matrix will be used as keywords for the continuing search. Differentkeywords/search terms will be selected based on the different filters orcombination of filters used. The entries for such filters may be machineor user defined. The combination and selection of different filters maycreate different permutation and results. The filters may include the“Word Lists”, “Phrases list”, and “Word Group Lists” which limit therange or area from which the keywords may be selected. The idealcombination of filters to control granular content for automaticproduction of keywords may be found also by trial and error. Artificialintelligence tools may be integrated.

Linear adjacent selections may be used as keywords. However, unknowncontent can slip through this sieve without an explicit humanintervention, therefore the Secured Metasearch Engine automates thissieve by creating permutations of both categorized and unknown words andphrases. These permutations are used as new search terms. The likelihoodof random combinations (included in the class of the NP! problem space)is so low, that emphasis is focused instead on linear adjacentselections. This result in is a small and focused problem space that canbe solved on a typical personal computer and ads disproportionate value.For example, a Google search of “Mohammed Atta” returns “Khalid SheikhMohammed” as a disconnected string of categorized and unknown words outof the box; that means parts of the phrase “Khalid Sheikh Mohammed” arerecognized without any special configuration, but not as a contextualphrase. Permutations of linear adjacent selections do however create thecomplete connected phrase and automatically submit the complete phrasefor a divergent metasearch, thereby creating new and previouslyunrecognized relationships to connect the-dots.

The engine can be set for automated contextual pruning with flexibilityto discover codes. Many divergent mining and search efforts will exposeunknown, deceitfully hidden, and statistically irrelevant relationships.For example, a Google search of “Mohammed Atta” also shows meetings atAleppo. The relevant thread for Aleppo is that it is Syrian universitytown with social connections to Hamburg and the 9/11 hijackersassociates. However, Aleppo is also a common Lebanese street name, andthe original shoemaker character in the Pinocchio story by C. Collodi.The name was changed to Geppetto in the Disney children's movie. Whileit is tempting just to prune this track, it is also important to exploreassociative contexts, because the use of codes can mask more sinistermeanings. The 9/11 hijackers communicated within the seemingly innocuouscontext of weddings, guest lists, and dates. So while contextual pruningis easier after the fact, it can be very important before the fact whenthe use of codes and false contexts are not so clear.

In conclusion, although stovepipes and silos, independent efforts atinformation collection and intelligence analysis, and air gaps orsecurity guards among domains prevent information sharing, the SecureMetasearch Engine enables security with information sharing includingMulti Level Security (MLS) search capabilities across domains. Itovercomes security leaks consistent with the needs to both protectnational security interests but also enable cross-domain informationsharing. The Secured Metasearch Engine technology is based on granulardata control. It enables secured searches and distribution of securedsearch results including secured information sharing with Multi LevelSecurity (MLS) search capabilities across domains. Data is explored notonly by content but also context and concept. Search should occur acrossdomains, with information shared and delivered across domains andthrough the extraction and integration of both private and public datasources including the Internet.

D.1 SECURE EDITOR, DOM EDITOR (GENERAL) AND DOM EDITOR—BASIC APPLIED ASEXTRACTOR MODULES

The Secure Editor can be used to generate SC or sec-con data for asearch and can also be used to locate and extract security or sec-condata and SC data before storage of the sec-com or SC data in thedistributed storage sites. The secure editor in FIG. 52 diagrammaticallyillustrates an editor which may be employed to secure sensitiveword/objects in a source document. In a current working embodiment, thesecure editor is a standalone application or a module to add into otherapplications for plain text and media creation, editing, and sensitivitylevel tagging. Other types of tagging, wherein the editor supplementsthe initial group or subset of security sensitive (or select content)words, characters, icons and data objects by categorization, taxonomyclassification, privacy, security, compliance, and semiotic meaning, arealso available. The editor supports a full range of document managementand can be integrated into a unified infrastructure, from creation,editing, document markup, tagging, tag conversion, tag removal, contextsensitivity level redaction, context reconstitution, and support forcomplex process work flows. The architecture assures separation of datafrom metadata so that no security lapses are introduced into thetraditional word processing and document management cycle.

FIG. 52 is a basic flow chart for one embodiment of the Secure Editormodule. Editor program 972 a begins with obtaining the source document974 a. Of course, the source document may be any type of document asexplained later herein. Step or function 976 a obtains one or morefilters for one or more security or sensitivity levels. Step 978 screensor processed the source document with the filter(s). For example, thesource document in FIG. 15 in window 991 has text regions 993, 994, 995and 996. In step 979, the Secure Editor in FIG. 52 displays, in situ (inthe displayed document), the filtered identified material and conformsthe precursor document to the security level protocols for the systemwithin which the Secure Editor is employed as an information processingtool. FIG. 16 shows that the address data 993 is marked TS (top secret),region 994 is displayed in color A for TS coding (please note that theaddressee data may also be so marked) and is “red-lined” or struck out.Region 995 is displayed as presented in the source document and islabeled U (unclassified) and region 996 is shown in color B, is redlinedand is labeled S. Labels TS, S, C (classified) and U are the establishedsecurity labeling protocol used by the organization employing the SecureEditor. Other labeling schemes may be employed. Color is used to assistthe user to select (and in some non-standard cases, deselect) thesensitive data marked by the editor. Redline is used to inform the userthat the filter(s) will extract the marked data. Labels are used topermit the entity using the editor to employ standard tear lineprotocol. Any data beneath a security classification of the user isunder the tear line and the data is permitted to be distributed to thelower security cleared user. Of course, electronic distribution ofsecure data need not use the hard copy or print version of the tearline. However, this nomenclature referring to the tear line is used inthe prior art systems.

Step 980 a (FIG. 52 ) accepts the user's manual changes (typicallyupgrades) to the precursor document. These manual changes are displayed,redlined, colored and labeled. Step 982 a inserts the security label TS,S, C and U has discussed above. Step 984 a notes that the system takescertain meta data such as author, date-time, version history, changehistory, etc. and converts this meta data into ordinary text, marks thatdata at the necessary security level and labels the data. Step 986 apermits the user to add (or omit) placeholders into the final document.FIG. 17 shows blank spaces which may be replaced by placeholders asblack lines or as XXXXX symbols (or other symbols) wherein the sensitivetext is not shown but some replacement markers are shown. The byline inregion 1003 shows “sanitized document.” The byline 1003 in FIG. 16 liststhe security level and the color representation.

Step 988 a (FIG. 52 ) activates the filter, extracts the sensitive or SCdata and temporarily stores the extracted data. Step 990 a displays thefiltered document and the user may view the filtered document at eachsecurity level. Therefore, the user, before transmitting a secured email(or letter) may look at the source (FIG. 15 ), may look at the TS levelwithout the redline strike out but with security labels and colors, maylook at the T level revealing regions 996 and 994 but not regions 993and 994 (which are TS coded regions), and look at U versions as shown inFIG. 17 . Step 992 a disperses the extracted data and the remainder dataor disperses partial versions of the document (those partial versionsformatted and containing only data at or above the target security level(all TS level data (which includes TS, S, C and U data), or all S data(comprising S, C and U) or all C data and U data)).

One feature of the present invention is that in step 979 a, the securitylevel protocol determines whether single words are granularly classified(TS, S, etc.) or whether a line is classified, or whether an entireparagraph is classified (see FIG. B-8B). If a commercial/privacy filteris used to exclude all social security numbers, the organizationalprotocol is set at a granular level to exclude just social securitynumbers. Different group protocols use algorithms to mark, filter andextract adjunctive security sensitive (or select content) words,characters, icons and data objects near the target security sensitivewords, characters, icons and data objects. The sensitive words may besecurity sensitive (or select content) words, characters or data objectsdefined by compliance with law, regulation or policy, privacy, national,organizational or private security concerns. For example, Bin Laden isthe target sensitive word in region 994 in FIG. 15 and this classifiesthe entire paragraph as TS level in FIG. 16 . The other words in theparagraph are adjunctive word/objects.

Several steps are prudent to enable MS Office document protection. Thefirst step is to have a network guard (see FIG. 48 ) that filters allincoming and outgoing traffic for MS Office document files andquarantines them. Spam and virus filtering is necessary to precludesystem, resource, and file exploits. URL filtering, quarantine lists,black lists, white lists represent the minimum responsible approach.Ingress files can harbor viruses, etc. Output or egress files can harborprivileged information at any and all levels of the DOM. With a means tofilter and check every node for purpose, content, metadata, formats,structure, comments, links, and so on, there is no other way to vet theintegrity of the file.

FIG. 53 shows the General DOM Editor program 1022 a in a flow chartform. Step 1024 a is the initialization that is employed by the securityentity to set up the program. Step 1026 a obtains the DOM layout orblueprint from the DOM vendor. Step 1028 a notes that for each root,branch and leaf in the DOM, the information attributes must beprioritized with th the security organizational informationalattributes. For example, the audio files of potential Bn Laden voiceprints may be critical at TS and S levels but be completely excluded atC and U levels. Therefore, any audio files below C and U are securityintrons which are excluded or ignored.

Step 1030 a obtains the source document DOM. Step 1032 a maps the DOMsource document as a binary file populated with content data and metadata as a subset of the DM blueprint. Step 1034 a identifies securityexons to be included in the further processing of the item andidentifies security introns to be ignored or excluded in the processing.Step 1036 a converts security exons at each root, branch and leaf intsecurity safe form and format (for example, a safe DOM template), andgenerates a safe blueprint or map for the precursor DOM and discards allsecurity introns. Step 1038 a processes the content of the safe DOM withsecurity filters discussed above. It also processes all data objects(exons) from the safe DOM with granular filters. Data is dispersed asnecessary and retrieved and re-assembled as necessary with a safe map.

FIG. 54 shows a flow chart of a basic application for a DOM Editor forMS Office document. Step 1042 a initializes the system. Step 1044 aobtains the source DOM document and notes that all processing occursafter a spam and virus filter. Step 1046 a notes that the programcreates or is provided with a security safe DOM document and map. Step1048 a notes that a template is opened. A clean .DOT template (Word) orwhatever the new document type is opened for the specific application.The Normal.DOT or New Spreadsheet.XLS on the MS Office distribution CDis safe. In Visio, for example, start with a new chart. Older documentssaved as templates tend to accumulate unanticipated metadata. If onemust use a non-standard template, clean it up. View it as both a printeddocument, as a binary file, and as a document object. Write protect thetemplates, or store the templates as non-modifiable networked volume. Ifthe templates are used throughout the organization, create a networkstore for them and configure each user's installation of MS Office tolook there for the templates. Search the network for duplicate copies ofthese templates and eliminate them.

Step 1050 a disables edit controls and step 1053 a copies content. Thepoint is one must edit cleanly. This is not a single step but rather aprocess, both one time and ongoing. Disable versioning in step 1050 a toprevent a buildup of past versions of the document. With versioning,prior sessions will be stored as document.doc 1, document.doc 2, and soon. These tend to get entwined with the latest version. If workflow withInfoPath, Outlook, or other collaborative workflow tools createsduplicate copies of the source document file for each user. Step 1050 aincludes the concept that the system is configured to store a singlenetwork copy instead. Preserve an audit trail and backup with a systembackup rather than versioning. Disable change tracking in step 1050 a tocurtail the buildup of additions, deletions, and changes that transcendthe publishing intent of the document. If redlining is necessary,establish guidelines for periodically accepting changes to purge changelog. Use the command to save the document without the change log withFile/Save or File/Save As. Do not use nor rely on fast saves, timedsaves, or file recovery after a MS Office crash to purge the dirtymetadata. After a crash, reopen the document, save the document under anew name. Close the Office application. Delete the old fileprecipitating the crash. Rename the new file under the old name. Reopenthe Office application.

Step 1054 a locates text in footnotes, hidden text, etc and eitherdiscards or ignores the subtext because those items are consideredsecurity introns or copies them into a safe DOM text form and disablesfunctions, if necessary. Step 1056 a notes that a clean map for thesecurity cleared DOM document must be created. Step 1058 a executes theeditor or granular filter and dispersal routine as necessary. Thedistribution or push of partial security cleared versions of the textcan be included in step 1058 a.

D.2 APPLICATIONS OF THE SECURE AND SC DATA SYSTEMS

The secure data storage systems in Part B, combined with the searchengine or KE engine in Part C can be deployed in various applications,some of which are described in this Part D.

D.3 AUTOMATIC FEATURES

The system and method described herein may operate substantiallyautomatically, that is, without operator intervention, other than thesecurity clearance function. The clearance function does require sometype of operator authentication prior to retrieval of the extracted andremainder data.

The system and the method may operate automatically in that theplaintext or originating data could be identified by a party desiringsecurity. The system could obtain that data from any data input device(hard drive memory, floppy drive memory, flash card memory, personaldata assistant (PDA), or any other type of data input device), filterthe data, separate the extracted text or the remainder text, encrypt (ornot encrypt) the data, separately store the extract and remainder data(all automatically, that is, without operator intervention). Hence, itis not necessary that the system operate with significant operator ormanual intervention. Of course, the system may also operate on aplaintext document or data object that is being created “in real time”by an operator and keyboard, mouse or other type of data input device.

The automatic operation of the system and the method can be caused by atriggering event. This triggering event may be a security attack(generating a trigger to start the gathering of plaintext, filtering,extraction and storing) or may be any other type of trigger such as abuilding burglar alarm, door alarm, fire alarm, or virus detectionalgorithm trigger. The event may be a time of day, week or month. It maybe n seconds after the user stops typing on a keyboard. It may be atimed back-up feature.

D.4 MULTIPLE SECURITY LEVELS

Multiple filters may be utilized in the system and in connection withthe method. These multiple filters may be useful in the operation of thesystem with a plurality of security levels. Each filter could filter outdifferent levels of security sensitive (or select content) items andeach bundle or group of security sensitive items (from each distinctfilter) could be stored at different computer storage locations.Multiple filters, multiple security levels and multiple storage areasmay also include multiple encryption routines and decryption routines.Encryption and decryption routines can be related to the level ofsecurity of a particular group of data.

Multiple maps may also be provided for singular or multiple storage ofextracted data and remainder data. These maps may or may not indicatethe originating point of the data. Maps can be parsed such that anintruder, upon discovery of a single map or map portion, could notlocate the storage locations of all piece of the extracted data andremainder data. Maps may also be encrypted. The map may also be storedat a distinct map store location.

The concept of partial reconstruction also includes the concept that aportion of the plaintext would be reconstructed and the unreconstructedportions of the plaintext could be encrypted or could show blanks orother symbolic indicators. See the placeholder table above. Partialreconstruction of the plaintext also includes a concept that thesecurity sensitive (or select content) items or materials may be subjectto different types of encryption. Hence, a single plaintext document mayhave multiple levels of security and multiple levels of encryptionwherein each encryption has a different level of security assigned toit.

The present invention can also be configured to provide a computernetwork which transparently establishes and manages the separation ofuser-based communities of interest. The separation is accomplished byextraction pursuant to security levels, dispersion of data into securestorage facilities (memory stores) and reconstruction based upon theassigned security level. A low level security clearance results in onlypartial reconstruction of the plain text or source document. Theseuser-based communities of interest are a plurality of users each havingrespective security clearances. As described above, each successivelyhigher level of security clearance permits the user to see greaterdegrees of reconstructed plain text obtained from the extracted datastored in extract stores and the remainder data from the remainderstores. By integrating encryption (and necessarily decryption),separation of user-based communities of interest are established suchthat the users in a particular community are permitted access to some orall of the plain text data based crypto-graphically separatedcommunities and need to know security levels.

Error

FIG. 30 is an exemplary computer network diagram showing various usercommunities. It should be noted that select content SC data stores maybe grouped for different portions of the same enterprise. Therefore,rather than have security groups as discussed herein, the system couldprovide SC interest groups in an enterprise. An “SC interest group” isthe same as a security level group. The telecommunications network 402is connected to the server application server provider ASP 452 and tovarious networks and personal computers or PCs. The PCs may be computerwork stations. Network A 404 is coupled to telecommunications network402 via an input/output unit 406. Network A is coupled to various PCsidentified in FIG. 30 as PC-4, PC-5 and PC-6. Of course, Network A couldbe coupled to other PCs not illustrated in FIG. 30 . As describedearlier, server 452 can facilitate remote or offsite storage of extractdata and remainder data in store 1, store 2 and/or store 3. Further, themap showing the storage location may be encrypted and stored in any oneor more of these stores. Also as described earlier, the memory in one ofthe PCs, for example PC-4, PC-5 could be utilized to store extract dataand remainder data from PC-6 and PC-6 can be configured as the inputdata computer. Hence, the present system and methodology encompasses theconcept of local storage and remote storage. On the local level, thestorage begins by storing the extract data at different locations in thehard drive of the PC. The next level higher is storing the extract datain removable computer media such as floppy disk, removable tape drives,CDs etc. associated with the PC accepting data or associated with aserver on Network A. The next higher level of extract store is storageof the extract data on a server or other computer in a particularnetwork. If PC-6 is designated as the input computer, the extract datamay be stored on PC-4. Of course, PC-4 could be designated as the serverfor Network A.

PC-7, PC-8 and PC-9 are coupled to telecommunications network 402.Network C 408 and Network B 410 is coupled to communications network402. The lines, one of which is line 409 extending from Network C 408,represent a plurality of computers or workstations coupled to Network C.Line 411 represents a plurality of workstations or computers coupled toNetwork B 410. In an e-mail implementation of one embodiment of thepresent invention, PC-7, PC-8, etc. may represent computerized devicesaccepting e-mail (personal data assistant, pager, cell phone, etc.). Thesender and the e-mail addressee may utilize simple computerized systemsto communicated via e-mail. Further, the network may be anytelecommunications network including wire, cable, cellular, wireless,satellite, IR or RF systems.

FIGS. 31 a and 31 b diagrammatically illustrates a flow chart showingthe key component steps for the multiple layer security program for thecommunity of users. The “community of interest” system described hereinenables persons and organizations at the same security level to sharedata on a peer to peer level. Further the security or SC system mayoperate automatically, with respect to extraction, storage andreconstruction, such that the peer to peer dissemination of data objectsis quickly and readily available to all at the same or higher securitylevels. Step 420 initializes the program. Step 422 enables the user,administrator or system operator to designate multiple levels ofsecurity, that is, multiple words, characters, icon, data objects, orwhatever, for each security level and further to define encryption foreach security level. The designation step 422 also includes identifyingthe communities of interest and the particular security level andsecurity clearance or SC level for each community of interest. Oneexample of various security levels for communities is set forth below inthe Community Security Level Table which is keyed to the computernetwork diagram of FIG. 30 .

Community Security Level Table Security level Community Group High PC-7;PC-8 Medium high all high group plus Network B Medium all above plusNetwork A Low all with nominal clearance Special set medium PC-7; PC-9;Network B

Further, designation step 422 will include identifying the words,phrases, icons or data objects subject to security or SC concerns andthe potential location of the extract data and, if necessary theremainder data and the degree of encryption. The following SelectionTable provides some examples.

Selection Table Level of encryption/storage - type or category of wordor phrase; input specific word, phrase High, web-based storage - dollarvalues, names of streets, countries, “Smith” and 5 words about  “Smith,” “avocado” Medium high, remote storage  - all addresses, allnames Medium network storage  - all family names, all client names Low,encrypt and separate - all items not in dictionary  store in localmemory

As an example of various encryption methodologies, the followingEncryption Table is illustrative.

Encryption Table DES, random pad A (“r. pad A”) Huffman, r. pad B CryptoAPI, r. pad 7 Two fish, r. pad C-2 Blowfish RC4 Skipjack Ghost

In FIGS. 31 a and 31 b , step 424 executes or enables the securityprogram or SC levels with multiple filters, multiple encryption levelsand multiple storage levels. Each one of these filters, encryptionlevels and storage levels correspond to the security level for thevarious communities of interest. Step 425 responds to an inquiry from auser to reconstruct the document. Step 426 accesses the user's securityclearance and the particular inquiry. Decision 428 determines whetherthe inquiring party is entitled to full or partial access to the sourcedocument. If not, the NO branch is taken and the system, in step 429adds placeholder substitutions. Step 429 may be optional. If YES, thesystem reconstruct pursuant to the clearance level in step 430. Thefollowing provides an example of multiple level encryption utilizingplaceholder substitution.

Example: Multiple Level Encryption

-   -   Applicants must be______zzxx xx______xxx______citizens and have        a high school diploma or equivalent. They must possess a valid        subsubsub driver's license and qualify for top        SUBWORD______clearance.

With this multiple level encryption, substitutions may be utilized“subword” to indicate to the user with a less than superior securitylevel that a certain word, term or phrase has been extracted and storedby he or she is entitled to know that substitute word, term or phrasehas been inserted into the plain text document. Of course, any type ofsubstitution character may be used for the placeholder.

In step 432, the system displays the plain text in a normal format orutilizing a split or bifurcated video memory or utilizing overlaydisplay screen. FIG. 7 and the description of that figure set forthabove describes the normal display in steps 202, 204, the split videomemory display in steps 206, 208, 210 and 212 and the overlay displaysystem in steps 214, 216, 218.

The system, in step 434, monitors and logs the location of the usermaking the inquiry, the type of inquiry, the time, day, date, clearancelevel and access level and logs all modifications to the plain textsource document. One example of the log is set forth below in theSecurity Report Table.

Security Report Table Privacy Scrubber Report  source file:path\filename  scrubbed file: path\filename-scrub  source file: date,time, size  process: date, time  user: name  system: name Recovery File (a) storage location, type of encryption, random key  (b) storagelocation B....  (c) store C ........  (d) store D ........

Step 436 enables the security program and parses and extracts the dataper the security program, filters the data, extracts it and codes itdisperses it and stores it as discussed above. The multiple layersecurity program ends in step 440.

The following Security Level Access Placeholder Table is another exampleof the type of placeholder substitutions that may be available. Theexample in the Security Table Access Placeholder Table may be used inconjunction with step 429.

Security Level Access Placeholder Table [security level 2] intelligencelocated [security level 4] 20 miles from [security level 4]. He is usingthe name [security level 4], and dressed as a [security level 4]preacher. With him are his lieutenants, [security level 4] and [securitylevel 4]. He is communicating with the international media through Mr.[security level 4], who resides at [security level 3], [security level4], [security level 4]. Telephone is [security level 1] and Facsimile is[security level 1].

It should be noted that in order to reconstruct some or all of the plaintext source data, some or all of the subsets of extracted data from theextract stores will be utilized dependent upon the respective securitylevel of the inquiring party or user.

D.5 SHARING DATA WITH DIFFERENT SECURITY LEVELS—DATA MINING

The present invention can be configured to overcome obstacles tointelligence sharing and data sharing between parties by enabling theparties to identify granular critical data and control the release thegranular critical electronic data subject to a sharing arrangement withother parties. In some instances, the controlled release process isdesigned to implement an agreed upon plan to share secured data or SCdata based upon arms length negotiations between the parties. Theinvention enables a party to release specific granular data such as aname, address, or date without releasing the entire “classified”document. In a commercial context, this is akin to data mining in thatthe inquiring party seeks limited data (not the entire data file, recordor document) and is willing to pay for the “mined” data. As an exampleof a security intelligence system, a local police chief may releasegranular critical data about a suspect to a federal agency, when inreturn the federal authority will release further intelligence “mined”or obtained for the secured data storage, about the suspect. Thecontrolled release of data from the higher security level party (theFBI) may be an intelligence document or a granular part of it (a partialreconstruction provided to the local police). The rationale behind thisimplementation of the invention is that there are many obstacles forsharing intelligence and information. There are even many more hurdleswhen it comes to sharing of raw intelligence. The invention creates aleveled playing field in which the different parties must share andexchange information in order to achieve their objectives.

The invention can be configured to resolve the major challenges facinggovernment by enabling sharing of information between its differentorganizations in relationship to fighting terrorism. The invention forexample can enable organizations, connected to the Homeland SecurityDepartment, to search data bases of various other government, state andlocal organizations, eliminating the fear of the “source” organizations,owning or controlling the source or plaintext documents that theirproprietary data or granular critical data is released without theirspecific permission. The invention enables open negotiations between theparties regarding what data to release and for what consideration. Whenseveral organizations are seeking access to a specific document, theinvention and can allow a controlled release of different granular datato different parties for different considerations and benchmarks.

The invention's mechanism of controlled release of the locateddocument/data enables other parties to search their documents withoutthe fear that sensitive information will be released to the searchingparty. This invention is designed to foster sharing of documentationbetween different parties, taking into consideration the need to limitthe access of other parties to the total content of the owner'sdocument.

The invention is a machine and process and its purposes and advantagesmay be as follows: (a) To automatically control selection of dataobjects within a data stream and release them in a controlled methodonly to authorized parties. (b) To automatically separate data objectswithin a data stream into two or more digital data streams according tothe importance and categorization of contents, through extraction andremoval of the prioritized content and its replacement by appropriateplaceholders. (c) To automatically control selected contents in E-mail,and enable its release in a controlled method only to authorizedparties. (d) To enable users to leverage the growth in computer andtelecommunications connectivity and electronic commerce by reducingsecurity risks. (e) To enable users to release documents, digital files,and data streams into closed and opened digital networks with theconfidence that important, identifying, and critical contents in thatdocuments, digital files, and data streams is secure and will be seenonly by authorized parties. (f) To enable real time simultaneouscustomization and personalization of selected contents within a datastream to different parties, allowing instant display of the selectedcontent or part of it based on, and tailored made to the status of theuser or receiving party. (g) To secure the important and criticalcontents of a document or digital file by transporting said contentsinto a separated data stream and removing said data stream to a removedstorage memory, while eradicating any copies, temporary caches, ortraces of the removed extracts on the original computer or machine. (h)To enable instant return transfer to the display or to another displayall or part of extracted content instantly with verification ofauthorized user. (i) To create a projection of the original document,digital file, data objects within a data stream, or variations of itthrough combined projection of the splinted data streams, whilemaintaining separation between the data streams. (j) To create analternative method for security, instead of encryption, which is secure,cost effective, less time-consuming, and flexible. (k) To enableautomatic timed removal of specific content items, automatically ormanually selected from a document, digital file, or data objects withina data stream. (1) To enable an automatic timed reconstruction(reconstitution) of the said document, digital file, or data objectswithin a data stream.

Another object of this invention is as a system and method forautomatically creating customized and personalized versions of adocument, data object, or data stream. In real time, simultaneousversions of the original are created and altered, then disseminatedbased on the status of the different users and their access privileges.The system and method enables content management and control byautomatically locating content items prioritized by importance,transporting them to a secure memory, and releasing them under explicitcontrols or preset rules.

Another object of the invention is as a system and method for control,analysis and management of important and prioritized information withindocuments, files, data object, and data streams. The system and method,enables the processing of all data objects at the time in which they arecreated or imported into the system. The early stage processing, enablesearly stage inventorying of prioritized contents as well as early stagepattern recognition. Extracting critical information, such as creditcard numbers, last names, first names, social security numbers, phonesnumbers, transaction dollar amounts and addresses, enables the systemand method to aggregate data in categories and analyze the data indifferent optional methodologies including pattern recognition.

Another object of the invention is as a system and method forcomprehensive monitoring of various activities including businessactivities in real time. With this level of detail, the system andmethod becomes a management information tool and information/datacommand and control center. The said system and method can include analert system, which in effect creates a real time apparatus for commandand control of the systems activities. In real time, and at any point intime, the user can get a comprehensive view of different activitiesincluding: (a) How many transactions are being processed, their content,their context, identity of the involved parties identity, theirprofiles, and the personnel involved. (b) How much money is beingtransacted. (c) When, in terms of dates, relevant to the transaction.(d) Where, in terms of geographical location, the transactions aretaking place. (e) Where, in terms of geographical location, monies orgoods are being transferred. (f) Which departments in the organizationare involved.

D.6 MULTILEVEL SECURITY THROUGH SANITIZATION WITH RECONSTRUCTION OFSANITIZED CONTENT

A multilevel security (MLS) or SC level technology secures the targeted,filtered content with extraction and dispersal to storage, bypassing theuse of classification labels, in order to achieve stronger security ofthe source document or data or better understanding of the SC material.During the process of developing security or SC technologies fordefending critical infrastructure, it was discovered that the businessmodel was too complex and there was a need to redefine and create newsystems and methods for doing business. As a result, one aspect of thepresent invention provides a system and codifies methods and businessprocesses to automatically identify, extract, store critical data (as aninput security system) and permit reconstruction of critical data onlyin the presence of certain security clearances (as the output of thesecurity system) or as established by the SC policy release rules.

The invention is a method and process to establish a stronger multilevelsecurity (or MLS) architecture and product, than is currently available.The invention introduces multilevel security through sanitization ofcritical content of a source or plaintext document (or data object) withthe unique ability to reconstruct all or part of the original documentin conformance to the classification level of the user. A user with topclassification may view the entire document, while a user with a lowerlevel classification will view a sanitized document, tailor madeautomatically for his clearance level. The invention secures thetargeted filtered content of a document, file, or data stream, throughextraction and dispersal to storage, bypassing the common use ofclassification labels in order to achieve stronger security. Theinvention enables secure document storage and secure message transfersbetween users and networks with different security classification levelswhile protecting the information on a need to know basis.

Currently multilevel security MLS systems are using multiple PCs foreach user, and using physically separate systems for processing data ateach classification level. The inventive system, in several embodiments,eliminates the need for the use of multiple computers. All the documentsin the user's PC are automatically secured with a granularclassification process generally described above with identification ofspecial security data, extraction from the source document or dataobject, and then separate storage of the security data. The classifiedgranular content is dispersed to different secure, distributed storagelocations. The classification level of a user will determine his rightand ability to access and release the stored critical extracted contentfrom the various storage locations for reconstruction. A user with topclassification will view the entire document, while a user with a lowerlevel classification will view a sanitized document, tailor madeautomatically for his clearance level.

Types of government security levels are: Top Secret (TS); Secret (S);Confidential (C); and Unclassified (UC). Business identifies securitylevels as: Restricted to Management (R, for example, attorney-clientprivilege); Proprietary (P); Sensitive (S); and Public (P). These MLSsecurity levels may be supplemented with “need to know” classificationlabels, organizational limits (Army, Navy, DoD) and time limits. Priorart security systems identified each file with: owner, size, date andtime of creation and security attributes. The Bell Lapadula (BPL)security model uses concepts such as domination of the MLS securitylevel over both a process and the subject (a data object). Some examplesof various processes are read, execute, overwrite, append, write, kill(delete), etc. Some examples of process rules under the BPL model are:NRU—No Read Up (a lower security level cannot read a document at ahigher security level); NWD—No Write Down (a higher level cannot writedown to a lower MLS level).

The invention herein does not use the “classification labels” of theprior art. Instead, it creates a situation in which the user gets accessrights to specific distributed storage locations based upon his MLSlevel, each access right can be classified with a differentclassification level. With respect to the editor described later herein,security labels (for example (“e.g.”), TS, S, C and UC labels) are addedor inserted into the filtered but not disassembled document. Asexplained later, the insertion of these SL labels conforms the currentinventive system to the prior art methodology and protocol. However, thecurrent inventive system does not use the SL labels for processingpurposes. The current system uses a granular or filter approach to makesecure the sensitive data in a particular document. SC labels, matchingthe relevancy of the SC data may be employed rather than security leveltags.

FIG. D-7 b diagrammatically illustrates a multiple level security systemaccessed by users having different security clearances (which alsorepresents a data mining system and operation). A MLS SC data systemuses different SC policy rules permitting access to different SC levelsof interest. Source data 100 passes through security program 200.Critical, important data objects or elements are extracted and dispersedinto storage 801. In the illustrated embodiment, storage 801 has foursecurity levels SL1-SL4, level SL4 being the most secure data requiringthe highest security clearance. Levels SL1-SL4 may refer to SC relevancylevels or organizational levels. Between each level is an MLS ormultiple level security guard. The guard (physical or softwareconfigured) limits transfer of data objects there between. Upon arequest or inquiry from user 1, 2 or 3, each having a security clearances1, s2 or s3, respectively, the query or request for access to data Q1,Q2, or Q3 is sent to security clearance process 803. The process 803detects and confirms the user's clearance level and passes a clearedquery to storage 801. Cleared data (an entire document/data object or aportion thereof or simply one secured data (i.e., a name)), is sent asData 1, 2 or 3 to clearance process 803. If clearance is still valid,data 1, 2 or 3 is sent to the respective user.

FIG. 32 can be a data mining system in that the user is permitted tomine the “cleared” data from storage 801. Data mining may be a monetarycharge associated with the clearance function in process 803.

In a secured system, the documents in the user's PC may be in“declassified” to his security level 99.9% of the time. The“declassified” or available documents are reconstituted through acontrolled release of the critical data from storage, and re-classifiedonly when the user presents his identification and his classificationlevel is being verified. Reclassification is automatic at the user's PC.The result is that the user's PC or workstation can operate inclassified and un-classified modes. It is unclassified when thedocuments are declassified and when the documents are reconstituted theuser is working in a classified mode.

The invention introduces a new paradigm whereby computers that areclassified as secret or top secret, in actuality will contain 99.9% ofthe time declassified documents. This capability strengthenssubstantially the security of such classified systems.

The invention can resolve the major challenges facing government inenabling sharing of information between its different organizations inrelationship to conducting military operations as well as fightingterrorism. The invention for example can enable organizations connectedto the Department of Defense (DOD) or the Homeland Security Departmentto search into data bases of various other government, state and localorganizations, eliminating the fear of the organizations owning thedocuments that their proprietary data or granular critical data would bereleased without their specific permission. The invention's mechanism ofcontrolled release of the located document/data enables other parties tosearch their documents without the fear that sensitive information willbe released to the searching party. This invention is designed to fostersharing of documentation between different parties, taking intoconsideration the need to limit the access of other parties to the totalcontent of the owner's document. The invention enables overcoming theobstacles of existing multiple level security MLS systems by enablingsharing of sensitive data, and granular data between parties in a muchmore flexible way which also enables much greater access to informationnot enabled by the current MLS systems. The invention includes acontrolled release mechanism for release of data in conformance tobenchmarks, which can include submitting of access identification, thegiving of consideration, submitting of other information, etc.

The invention creates better collaboration between users andorganizations based on a better flow of information. It enables betterefficiency enabling easier communication between users and networks withdifferent levels of classification while maintaining the highest levelsof security. The invention enables a much better management of documentsin storage and in transport including e-mail. The invention introducesautomation to the sanitization process and an automatic reconstructionprocess. The automation will avoid human error both intentionally aswell as unintentionally. The automation will enable a substantialreduction in costs, furthermore the ability to create a multilevelsecurity environment in one PC or workstation will save costs ofpurchasing operating and maintaining multiple machines as is the currentpractice.

The challenge of many organizations is in getting mission critical andtime sensitive information speedily to the users who need it. In manycases the needed non-classified or low-level classified information isstored in systems but is not provided to the user who needs it, becausethe information is in documents which are highly classified. Thiscreates situations in which users are unable to access information,which they need to accomplish their tasks, because of a technologicalclassification barrier. This over classification of information resultsin hampering critical tasks and activities, as well as creating systemredundancies inefficiencies. The DoD (Department of Defense) multiplelevel security (MLS) was based upon the Bell-Lapadula (BPL) Model. Manybelieve that the BLP security model is superior to other models. TheBell-Lapadula Model and the existing MLS uses labels to classify usersand subject matter. A professional attacker will use his efforts tochange or damage the labels in-order to compromise the machines securedinformation.

The architecture or the present invention extracts and physicallyseparates data whereby content is being recognized not by labels byautomatically based on the semantic content of the plaintext. In someembodiments of the present invention, labels are added to the granular,filtered document to conform to known organizational protocols. Theselabels are displayed but not used by the system in processing.

In the DoD's MLS, data of multiple security levels are processed andtransferred by the system, which separates the varying security levelsand controls access to the data. In the prior art MLS system, someapplications process only one level of data at a time, (for example,when a user edits a document with a word processing tool, the data inthe document are treated as if they were a single level, theclassification of the document itself). Other applications treatindividual data elements at their actual levels. For example, a wordprocessor enforces paragraph and page MLS classification labels, or anMLS data base brings together data elements of different security levelsto allow an analyst a multilevel view of the information.

The vulnerabilities of MLS: The components in the MLS system contain thedata in their memories and disks, and the data could be compromised ifadequate physical security is not maintained. An attacker who getsaccess to the system might be able to locate the data or its copies. MLSguards control the flow of information across security boundaries. TheseMLS guards are known. One concern with the Bell-Lapadula Model and theexisting MLS is the use of labels to classify users and subject matter.A professional attacker will use all his efforts to change or damage thelabels in-order to compromise the machines secured information. Theinvention introduces an architecture whereby content is being recognizednot by labels by automatically based on the semantic contents of theplain text. The invention sanitizes and enables reconstitution uponvalid authentication. It is the only architecture and system whichenables both sanitization and reconstitution according to user'sverified access identification. The conventional way of classifyingdocuments with high classification (TS), limits the low level clearanceusers (C) from accessing substantially un-classified information“granular data” which is in the classified document. Furthermore, theinvention enables maximum sharing of unclassified information which liesdormant in classified documents. Top security-secret information isdispersed to distributed storage in many locations. The invention isdesigned to avoid any one point of failure. The theory behind thearchitecture is the creation of substantial lines of defense in depth.The attacker will need to break through many obstacles before accessingall the dispersed data of the document. Additional levels of securityare provided with multi-type encryption. The system and processintroduces the capability to encrypt different parts of a document withdifferent types of encryption. Multi type encryption creates a majorbarrier to an attacker. Should he wish to break the encryption, he wouldneed many super computers. Should the attacker look for implementationmistakes, even if he finds few, he will still not get access to thetotal plain text. The inventive system provides flexibility. The systemand process delivers flexibility to accommodate changing circumstances.By controlling the level of the granularity, the user can boost thelevel of security according to changing circumstances. For example, if acompetitor becomes a partner the user enables him access to more storagelocations, by changing the matrix.

The system and process integrates the Internet for dispersal and hidingof contents. If a party needs more information, it could be releasedgranularly. There is no need to release the whole secret document. Thesystem and process does not use labeling but rather extracts thecritical to storage (bu the system may label sensitive text to conformto known protocols). The system avoids situations, in which, attackersmay manipulate the labels or the labeling system. Furthermore, therelease of information is based on changing circumstances (time,location-GPS, event).

The invention is a machine and process and its purposes and advantagesmay be as follows: (a) To automatically control selection of dataobjects within a data stream and release them in a controlled methodonly to authorized parties. (b) To automatically separate data objectswithin a data stream into two or more digital data streams according tothe importance and categorization of contents, through extraction andremoval of the prioritized content and its replacement by appropriateplaceholders. (c) To automatically control selected contents in E-mail,and enable its release in a controlled method only to authorizedparties. (d) To enable users to leverage the growth in computer andtelecommunications connectivity and electronic commerce by reducingsecurity risks. (e) To enable users to release documents, digital files,and data streams into closed and opened digital networks with theconfidence that important, identifying, and critical contents in thatdocuments, digital files, and data streams is secure and will be seenonly by authorized parties. (f) To enable real time simultaneouscustomization and personalization of selected contents within a datastream to different parties, allowing instant display of the selectedcontent or part of it based on, and tailored made to the status of theuser or receiving party. (g) To secure the important and criticalcontents of a document or digital file by transporting said contentsinto a separated data stream and removing said data stream to a removedstorage memory, while eradicating any copies, temporary caches, ortraces of the removed extracts on the original computer or machine. (h)To enable instant return transfer to the display or to another displayall or part of extracted content instantly with verification ofauthorized user. (i) To create a projection of the original document,digital file, data objects within a data stream, or variations of itthrough combined projection of the splinted data streams, whilemaintaining separation between the data streams. (j) To create analternative method for security, instead of encryption, which is secure,cost effective, less time-consuming, and flexible. (k) To enableautomatic timed removal of specific content items, automatically ormanually selected from a document, digital file, or data objects withina data stream. (l) To enable an automatic timed reconstruction(reconstitution) of the said document, digital file, or data objectswithin a data stream.

The invention differs from the current implementations of multilevelsecurity MLS systems based on the Bell-Lapadula Model, and the prior artuse of labels to classify users and subject matter. A professionalattacker will use all his efforts to change or damage the labelsin-order to compromise the machines secured information. The presentinvention introduces an architecture whereby content is being recognizednot by labels by automatically based on the semantic contents of theplain text. The invention enables overcoming the obstacles of existingmultiple level security systems by enabling sharing of sensitive data,and granular data between parties in a much more flexible way which alsoenables much greater access to information not enabled by the currentMLS systems. The invention includes a controlled release mechanism forrelease of data in conformance to benchmarks, which can includesubmitting of access identification, the giving of consideration,submitting of other information, etc. The invention creates bettercollaboration between users and organizations based on a better flow ofinformation. It enables better efficiency enabling easier communicationbetween users and networks with different levels of classification whilemaintaining the highest levels of security. The invention enables a muchbetter management of documents in storage and in transport includinge-mail. The invention introduces automation to the sanitization processand an automatic reconstruction process. The automation will avoid humanerror both intentionally as well as unintentionally. The automation willenable a substantial reduction in costs, furthermore the ability tocreate a multilevel security environment in one PC or workstation willsave costs of purchasing operating and, maintaining multiple machines asis the current practice.

D.7 MULTIPLE INDEPENDENT LEVELS OF SECURITY (MILS)

FIGS. 48, 49 and 50 diagrammatically illustrate a computer systemconfigured as a multiple independent levels of security (MILS) system.Although the general operation and layout of the MILS system is wellknown, the incorporation of the inventive system, that is, granularfiltration, extraction and re-assembly is unique to the inventivesystem. In the following MILS system 910, the dispersion and retrievaloperation of the present application is discussed using, as an example,one of the most common environments for protecting classified data, thatis, Multiple Independent Levels of Sensitivity (MILS). The MLS system,discussed in connection with security levels, can be implemented with SCbased upon either relevancy or based upon enterprise units who havepermission (policy rights) to access the SC data.

As is known in a MILS configuration, each level (TS—top secret;S—secret, U—unclassified) of classified data is isolated from otherlevels by confining it to set of components dedicated to a singleclassification level. Data labels are not used within the system, sinceits components are Commercial-Off-The-Shelf (COTS) products that are notable to handle data labels. The level of the system TS, S or U (topsecret, secret or unclassified) establishes an implied sensitivitylabel. Data is explicitly labeled only in guards 926, 932 (the Dispersaland Re-assembly guard computers) and other MLS devices that connect theMILS system to systems at other sensitivity levels, that connect the TSsystem to the S system and to the U system. Data transfer between levelsis accomplished manually (sneaker net), or through a few, high assuranceand closely protected MLS devices, such as guards, digital diodes, etc.A user with the authority to access multiple levels of data is requiredto use a separate set of interface equipment to access each MILS system.In some cases, keyboard-video-mouse (KVM) switches are permitted.

FIG. 48 shows an implementation of the present invention in a MILSenvironment. Two similarly configured domains 911, 915 are shown (upperregion consisting of sub-networks 912, 913 and 914 and lower network915), presumably under different administrative control. Each domain hasthree networks, one for each of three classification levels. Eachnetwork includes a plurality of workstations (only one station beingshown in the figure), a Dispersion and Reassembly (D&R) server 926, 932,(a MLS device connected to all networks), and database servers TS, S andU data servers (928, 929 and 930 in the upper domain and 934 in thelower domain) to support the D&R server's proxy document serverfunctionality.

The user interface components of the present invention reside on theuser workstations W St-U (920), W St-S (918) and W St-TS (916).Dispersion and reassembly functionality is hosted in the D&R servers926, 932. Digital signatures (encryption) protects the integrity of userdata between the user interface at 916, 918, 920 and the D&R server 926.Encryption provides a level of discretionary access controls thatprevents disclosure in cases where others have the appropriate clearanceut lack the formal “need to know” level to view the classifiedinformation.

The present example discusses an e-mail and a file server to helpclarify its operation in a MILS environment. The system may beconfigured to handle other documents, images, etc. In the e-mailexample, a Top Secret user 916 will prepare a multi-level message andsend it to a Secret user 922 b, and the recipient 922 b will only beable to read the “Secret Level” and below parts. In the file serverexample, a Top Secret user 916 will prepare a multi-level document andpost it to a file server, then a Secret user 922 b will retrieve it,getting only the Secret and below parts. These are both downgradingexamples. Movement of data within a level and to higher level conformsto existing security rules, for example, Bell-LaPadulla rules.

The path an e-mail message takes from a Top Secret workstation 916 inone domain 911 to a Secret workstation 922 b in the other domain 915. Itwould work similarly if both sender and receiver were in the samedomain; the local D&R server 926 would perform the functions of both D&Rservers 926, 932 in this example. The D&R server 926 hosts an e-mailserver that is customized to perform D&R functions, as described below.

An e-mail originator on a Top Secret workstation 916 composes an e-mailusing MS Outlook. Originator marks sections of the message with TS, S,and U levels (explained herein), and the system on workstation 916compartment tags the email document using a combination of the automaticfeatures described earlier and manual tagging (user based), as allowedby the security policy of the accrediting organization. Originator 916digitally signs and sends the message to the mail server on the D&Rsystem 926.

The Disperser component of the D&R mail server 926 verifies that theoriginator 916, identified by e-mail address and authenticated by thedigital signature key, has the authority to downgrade messages to therequested levels. The Disperser within Server 926 separates the messageaccording to the originator's tags. The Disperser writes the messagesections tagged higher than Unclassified to the Secret and Top Secretservers 929, 928. The unclassified part becomes the base message mapstored in server 930 and contains pointers to the higher-levelcomponents. The pointers map the re-assembly path when the document isfully or partially re-assembled. This base message map is the only stepin the process that requires downgrading. A guard 926 is used to provideincreased assurance that the downgrading is done correctly (see FIG. 49, guard 936). The originator's identity is provided to the guard as partof the downgrade credentials. The Disperser forwards the base messageand message map, to the D&R e-mail server 932 of the addressee, usingits Top Secret network path c. Other paths 924 a, b and c link the samesecurity levels in either domain. Using the TS path prevents downgradingif the target server 932 is not a D&R server. Alternately, the D&Rservers 926, 932 could authenticate each other before transferringmessages.

The target D&R e-mail server 932 determines the clearance of theaddressee 922 b from its network address and looks up the addressee'scompartment authorization in its own tables. It then removes links fromthe base message and map to all message components that are notdominated by the addressee (as defined by Bell-LaPadula). The target D&Re-mail server 932 then retrieves the data for the remaining messagecomponents from S database server 929 via guard server 926, constructsor re-assembles the message that the addressee is cleared to read(secret level, not top secret level), and places the message in theaddressee's inbox 922 b that is at the security level of the highestcomponent in the message. These steps are executed for each addressee.The addressees 922 b connect to their local D&R e-mail servers 932 anddownload unread e-mail.

similar method is used for document serving, such as would be used forfile service, web service, ftp service, etc. The document creators tagtheir documents, sign them digitally, and post them to a D&R file server926, 932, which is actually a proxy using storage in the databaseservers 928, 929, 930, 934, and others not numbered. The server 926, 932disperses the message components to its database servers on the MILSnetworks.

The following describes the path a document takes from its creation (ormodification) in a Top Secret workstation 922 a in one domain 915 to areader on a Secret workstation 918 in another domain 911. The D&R server932 hosts file servers that are customized to perform D&R functions, asdescribed. The document posting process follows: A document is createdon a Top Secret workstation 922 a using MS Word, Excel, etc. Theoriginator 922 a marks sections of the document with TS, S, and U, andcompartment tags using a combination of automatic and manual taggingfeatures, as allowed by the organization's security policy. Originatorat 922 a digitally signs and sends the document to the file server onthe D&R system 932.

The Disperser component of the D&R file server 932 verifies that theoriginator 922 a, identified by the digital signature, has the authorityto downgrade documents to the requested levels. The Disperser in server932 separates the document according to the originator's tags, creatinga base document at the lowest tagged level (U-db or S-db) that containspointers to the more classified components (TS-db 934). The basedocument is encrypted using an algorithm appropriate to its level. TheDisperser writes the document sections to the file servers (U-db or S-dbor TS-db 934) according to their tags. This is the only place in thesystem where downgrading is performed. A guard 932 is in the downgradepath to provide additional assurance that data is not leaked. The useridentity derived from the originator's 922 a digital signature on thetagged message will be provided to the guard 932 as part of thedowngrade credentials. The disperser places references to the documentin the directory of each file server 932, 926 for which a version of thedocument can be built (e.g., if there is no unclassified data in thedocument, a reference to the document will not be placed in theunclassified directory).

The document retrieval process follows. A user on a workstation 916 logsonto its local D&R proxy document server 926. The strength of userauthentication is application specific and determined during systemaccreditation. The user 916 locates a document and requests that it bedownloaded for reading, specifying a requested sensitivity level andcompartment. The Re-assembler component of the D&R server 926 loads anddecrypts the base document. The Re-assembler in server 926 verifies thatthe requestor 916 dominates the requested security level andcompartment, based on the level of the network over which the requestwas received and the contents of its compartment authorization table.The Re-assembler in server 926 constructs the document to the authorizedsecurity and compartment level. The Re-assembler provides the documentto the requestor. The re-assembly function does not violateBell-LaPadula and does not require downgrade authority.

D.8 MLS SERVER AND WORKSTATION COMPONENTS

FIGS. 49 and 50 diagrammatically illustrate the server and workstation.FIG. 49 shows the D&R Server 926 hosted on a DigitalNet CTS-400 system,which is currently evaluated. It consists of the CTX-400 hardware andthe STOP 6.0 operating system. The disperser part 937 of the D&R server926 has the most security critical functions, since it must move datacontrary to the Bell-LaPadula security policy. It maintains a table ofuser downgrade authorizations 938 keyed by the originator's publicdigital signature key 939 and downgrade authority 941. A guard 936 isalso included, which may employ a version of automated tagging system ofthe present invention to identify sensitive data. The double check ofthe security label tag for each part of the message/document provided bythe guard 936 is particularly important since the downgrade labels areapplied in a single-level system.

The re-assembly side (in re-assembler 942) requires no policy violation,but requires its own table of user compartment authorizations 944because the MILS systems do not have the ability to label data. Thetable 944 is keyed by e-mail address (for e-mail routing) or useridentity 945 from the user authentication process (for file service).Authorized compartments 946 are coupled to email user id 945.

The components of the user workstations are shown in FIG. 50 . A MILSworkstation 916 is a single-level component, that is, it is coupled to asingle security level network. The user interface will in some cases beinstalled on existing workstations 916 and in other cases newworkstations procured for this use. The workstations include anoperating system Windows 2000, a graphical user interface Windows GUI,WS Office as a document processor, digital signature system rated at EAL4 and a rated user interface.

D.9 FLEXIBILITY OF THE PRESENT MLS APPROACH

The late-binding techniques or retrieval and reassembly features used inthe present invention is a compelling feature, because it providessolutions to some of the most significant problems of sharing data inmodern warfare and international coalition operations. A singlecomprehensive document can include data sensitive to many differentenvironments, yet by tagging its components correctly, maximally usefulversions can be provided to readers with widely differentauthorizations, and without having to identify them all ahead of time.

For example, in a coalition environment, countries or classes ofcountries, or agencies within countries can be assigned non-hierarchicaland hierarchical labels and then a single document, suitably tagged, canbe securely distributed to all of them using a properly configured andadministrated infrastructure of the present invention.

In the presently configured MILS embodiment, the tagging protocol is, ata minimum, confirmed by the guard in the D&R servers. In prior artsystems, only the guard tags documents. In the presently configuredsystem, the thin client applications program on the workstation tags thedocument segments but the guard confirms this preliminary tagging. Onefeature which seems to be important in certain security systems is thatthe re-assembly map be cleaned or scrubbed to match the then currentsecurity level where the map is stored. This theory, carried forward,would include the concept that the re-assembly map, stored in eachversion of the secured document, only point to the next level storagelocation. Hence, if all portions TS extracts, S extracts, C (classified)extracts and U (remainder) document are dispersed into, for example TS928, S 929, C (not shown) and U 930, then the document in U 930 onlyincludes a pointer to the location of C extracts and the C extracts onlyincludes a pointer to the S extracts and the S extracts includes only apointer to the TS extract location. The downgrade discussed above refersto the location of the higher secured extract.

D.10 MULTIPLE EXTRACTION FILTERS AND APPLICATION OUTLINE

There is a need to construct filters which supplement the initial listor compilation of security sensitive (sec-con) or select content (SC)words, characters, icons and data objects (herein “word/objects”). Theneed arises either due to the fact that the initial security or SCword/object list is incomplete, or that the author of the initial listis concerned that the list is too limited or in order to defeat a attackor an inference engine “reverse engineering” the sanitized document andascertaining not only the filter (a type of code) but also the sensitiveword/object removed from the source document. Further, the incorporationof a filter generator enhances the current user friendliness of theprogram. In its current embodiment, the program is configured as aneditor to screen and sanitize a source document. The user selects, athis option, functional aspects which include: compliance with laws (anapplication of a type of filter, e.g. HIPAA, GLB, Oxley-Sarbanes, EUprivacy, executive orders); privacy (another type of filter whichexcludes, for example, social security numbers, see also, EU policy);search for and supplement filter; pay per view (which enables the userto buy missing sensitive information (for commercial purposes); survival(which creates a distributed and dispersed copy of the user's documentand other stored documents and items using predetermined storagefacilities); security (which triggers the various security routinediscussed herein); and storing (which permits the user to select whichof the several storage options the extracted sensitive or SCdata/objects should be employed in the dispersal.

The filter routine diagrammatically illustrated in FIG. 51 is useful incompiling a filter which separates both the sensitive word/objects andcontextual and semiotic and taxonomic aspects of the initial list ofsecurity sensitive (or select content) word/objects. The filter works inconjunction with a compilation of data, typically located on a networkwhich could be private or public. In low level security or SCsituations, the filter may access Internet databases to gatheradditional data for the filter. In more secure or SC systems, the filtercould access a secure or SC data base (one located at the same securitylevel as the user) and build or compile the additional word/objects. Thefilter program 950 a in FIG. 51 begins with step 952 a which compilesthe initial list of security sensitive (or select content) word/objects.In 954 a, the initial list is supplemented with dictionaries, phonebooks, corporate records (to obtain subsidiary data and trade names) andthesaurus data. Each of these represent different compilations of dataand the added data is added to the initial list of sensitiveword/objects. In 956 a, a search is conducted on a network, usuallythrough a search engine, to gather excerpts near and abut the keywords.These keywords are the initial sensitive word/objects. Statisticalalgorithms are applied to gather non-common word/objects which areassociate with the keywords as found in the additional datacompilations. The goal of the adaptive filter is to obtain contextual,semiotic and taxonomic words, characters or data objects from thecompilation of additional data related to the security sensitive (orselect content) words, characters or data objects. Semiotic is a generalphilosophical theory of signs and symbols (read language and words andobjects) that especially deals with their function. Semiotics includesyntactics, semantics and pragmatics. Syntactics is the formalrelationship between signs. Semantics is the meaning of signs andpragmatics is the relationship between signs and their users, such asthe relationship of sentences to their environment. Taxonomy is thescientific classification and categorization of items. Therefore, as anexample, a search through the Internet on Google search engine under“Bin Laden” may show a number of uncommon (non-dictionary words) within200 words of the target “Bin Laden.” This search string would gatherdocuments form the Google search and copy 200 words on either side of“Bin Laden” and then extract only non-dictionary words into asupplemental list. This type of filter algorithm looks for contextualmatters close or near to the target. The search is semiotic andstatistical in nature. Additionally, the initial supplemental list wouldidentify the Bin Laden is an arab and this classification (a taxonomicaspect) can be used to expand the list for the filter. The algorithm mayinclude a simple command to gather all 10 words on either side of BinLaden. This is a pure contextual search and the “10 word” aspect is astatistical number. From the supplemental list, all pronouns,prepositions and conjunctions may be eliminated. Spiders or robots maybe used in the gathering of the contextual and semiotic filter data. Thecontextual, semiotic and taxonomic words, characters or data objectsfrom the compilation of additional data is all related to the initiallist of security sensitive (or select content) words, characters or dataobjects.

Step 958 a compiles the adaptive filter. The above noted contextual,semiotic and taxonomic filter is adaptive since it can be used to expand(and potentially contract or reduce) and adapt an existing list ofsensitive word/objects to a larger list which better protects the sourcedocument and inhibits the operation of an inference engine. Step 959 arepeats the filter gathering and compilation for various levels ofsecurity. Higher security may require a broader search (1000 uncommonwords near Bin Laden and add all Arabic and sub-Asian continent cities).Orthogonal security groups (those groups having the same level, e.g., SSecret, with each other but being different organizations, e.g,Department of Defense compared to the FBI) often have different methodsto keep data secret between compartments.

The adaptive filter can be set to automatically gather additivesensitive word/objects. The system, with a basic filter, may identify asensitive word in a paragraph being scanned by the initial filter. Thissensitive word may be a special word in the existing filter or may be anon-common word not found in the initial filter. The adaptive filtersystem may then obtain this “unknown” or “special” word, and conduct asearch through a compilation or data base of additional words, etc. Anynew word/objects falling within the contextual, semiotic and taxonomicwords, characters or data objects from the compilation of additionaldata (database) related to said security sensitive (or select content)words, characters or data objects are then added to the filter. Theexpanded filter is then used to screen the source document.

Step 960 a compiles a supplemental filter with random words, phrases,etc. in order to further defeat an inference engine reverse engineeringassault on the secured and sanitized document. In some sense, theproduction and use of a random filter is an encryption technique sincethe resultant filtered product, in order to be understood by others,must be reverse filtered or decrypted to reveal the document at theappropriate security level. Nonsense words may be added to thissupplemental filter. Step 962 a applies the primary filter (with thesecurity word/objects and the additive word/objects from the contextualet al. filter) to the source document. Step 964 a extracts the sensitiveword/objects per security level. It is noted that several filters areused, on one for each security level, whether hierarchical ororthogonal. The extracted word/objects are stored or the partiallyextracted document per security level is stored in the correspondingsecurity cleared data base or storage. Step 966 a applies thesupplemental filter to the remainder or lowest classified document. Step968 a stores the supplemental random filter to permit the low level userto decrypt the document. Step 970 a publishes, distributes or pushes thedocument to others having a need to know. The pointer to the location ofthe supplemental filter decoder is encrypted and stored in the filteredremainder document. This permits the low level person to decode theremainder document.

D.11 MULTIPLE SECURITY FEATURES FOR DATA

FIG. 40 diagrammatically illustrates a flowchart showing the keycomponents of a multiple encryption program using multiple types ofencryption in one document or data object. Multiple levels, types ormodes of encryption are utilized in the same document or data object toenable securing data and transparently managing the separation ofuser-based communities of interest based upon crypto-graphicallyseparated, need to know security levels. These security levels areassociated with a plurality of encryption types or with different cipherkeys using the same encryption. An example of a multiple level encrypteddocument is shown above in the Multiple Level Encryption sample.Different levels or modes or types of encryption are listed in theEncryption Table above.

Step 510 in FIG. 40 initializes the system by organizing differentsecurity levels with different encryption types and cipher keys. Also,the program sets filters to create the multiple encryption or MLdocument or data object. Step 512 filters the document or data object.Step 514 encrypts the extracted data for each security level. Thesesteps 510, 512 and 514 utilize many of the routines discussed above inconnection with FIGS. 9 and 31 a and 31 b, steps 232, 234, 236, 238,240, 422 and 424. Step 516 recognizes that the secured document or dataobject may be stored for later use (with associated multipledecryption), published, distributed, or otherwise utilized to achievethe primary purpose of the document, i.e., to communicate information orto safely store security critical information. Step 518 permits theuser, with the proper security clearance to retrieve the document ordata object. Step 520 illustrates that the user must retrieve his or hercipher key to decode all or a portion of the ML encrypted document ordata object. This step may be manual which engages the user to intocertain codes or may be automatic such that the user's computerautomatically, without operator input, decodes all or part of thedocument or data object. Step 522 decrypts the document pursuant to theuser's security clearance. Step 524 recognizes that the user may review,re-publish, store, comment on, re-encrypt or otherwise deal and handlethe full or partially decoded document or data object. The program endsor otherwise continues with other programs set forth herein. It shouldbe noted that storage of the extracted data may be included in the flowpath of the program in FIG. 40 is necessary.

FIG. 41 diagrammatically illustrates a chart showing the key componentsof the parsing, dispersion, multiple storage and reconstruction (undersecurity clearance) of data. Document or data object 100, in functionelement 550, is created or obtained by the input computer device. Thedocument is stored in a normal manner in customary data store 552. Aparsing algorithm function 554 is utilized in parsing step 556. Theparsing algorithm, as stated earlier, targets the plaintext document ordata object 100 and splits, cuts and segments (that is, parses) thedocument by bit count, word, word count, page, line count, paragraphcount, any identifiable document or icon characteristic, or otheridentifiable feature such as capital letters, italics, underline, etc.Hence, the parsed document 100 constitutes at least remainder data anddata which is extracted or parsed or segmented out. A plurality of dataextracts may be obtained. The parsed data (which is both the extractdata and remainder data) is then dispersed into storage facilities datastore DS 1, 2, 3, 4, etc. Preferably, the parsed documents are encryptedas shown by “e” in FIG. 41 . In order to facilitate the potentialreconstitution of document 100, a map is stored in a map storage 558.Hence, the dispersement 560 largely spreads out or distributes theparsed document 100 to a plurality of memories in the distributedcomputer system. These memories may be removable memory devices (floppydisc, removable tape drive, CDs) or may be more fixed devices such ashard drives, Internet storage facilities, etc. Preferably, the map isalso encrypted.

Reconstruction step 562 enables a person with the appropriate securityto obtain the map from map storage 558, decode the map, gather thedispersed, parsed segments of document 100 and compile the document.This is noted in function 564.

Since the original document 100 is stored in a customary manner in datastorage 552, the parsed document stored in multiple data storage unitsDS1-DS4 provides a unique backup for document 100. The algorithm canemploy many different mathematical constructions but is, in the currentembodiment, primarily based upon one or more of a bit count, a word, aword count, a page count, a line count, a paragraph count, andidentifiable document characteristic, and identifiable wordcharacteristic, and identifiable icon characteristic and identifiabledata object characteristic, capital letters, italics, and underlinefound in the plaintext document or data object. Further, the parsingalgorithm can generate different security levels wherein parsed segmentsare stored at different storage facilities having various degrees ofsecurity clearance. This establishes a hierarchy of data storage unitsand corresponding degrees of security clearances. The parsing algorithmmay identify unique words or strings of data, i.e., credit card numbers.The hierarchy of security clearances may involve first a password,second a biometric confirmation such as a voice match and a third highlyunique biometric characteristic such as a fingerprint or retinal scan.The parsing system enables a large distribution of data in a securedenvironment. In this manner, if the original data object 100 atcustomary data storage 552 is destroyed, a person with an appropriatesecurity clearance can reconstitute the original data document 100 dueto the secured parsing and dispersal of document 100 through datastorage units DS1-DS4 and map storage 558. The parsing may occur on agranular level. In particular, the parsing may occur on a financialdocument in electronic form.

Financial Document Table Startcode; Abel, Robert, NMI; 100567;TRANSFER803; To8900586943; FROM3897622891; $700.00; end code

In the Financial Document Table, the start code and end code istypically represented by a digital code unique to the communicationschannel, the name on the account has no middle initial (NMI) and thevarious words “transfer 803” and “to 8900586943” and the words “from”and “$” are represented by predefined numeric or alpha numeric codes.The electronic financial document complies with an established protocol.In any event, financial documents are often times transmitted throughelectronic communications and telecommunications channels. The presentinvention, in one embodiment, enables a higher level of security byparsing the financial document or data stream. Further, a higher levelof security may be employed by extracting identified text or charactersand storing the extracted text as discussed above in connection withFIGS. 4, 5 and 6 .

To some extent, the present system can also be utilized for keymanagement and encryption systems.

In a broad sense, the parsing methodology disclosed herein is not basedupon the separation of critical versus non-critical or classified versusnon-classified security information. The primary focus of the parsingmethodology is (1) automatic transparent parsing of data content intogranular data groups which are thereafter dispersed to different storagelocations in order to maintain a very high level of security with orwithout encryption; (2) dispersal of the segmented data to differentstorage locations each which, potentially, demand additionalidentification or security clearance prior to the release of the storedsegmented data, including, possibly, the creation of a digitalbureaucracy, in order to hinder or circumvent digital attacks on theplaintext document or data object; (3) proposing and implementing asystem wherein the user has a very basic appliance since most of theuser's data is stored both locally (customary data storage 552; FIG. 41) and parsed and stored in a distributed system (DS1-DS4) and wherein animportant asset is the map stored in map location 558; (4) enabling aninstitutional system to parse highly confidential information andextract the same in granular form and disperse the same throughout theInternet or other storage locations with or without encryption withoutcompromising the document's security privacy and integrity.

The process involves parsing the documents or content into granular datagroups and optionally creating small groups of data wherein the datasegments cannot be recognized even to the level of providing 2-4 dataobjects in each file; dispersing the granular data groups into differentstorage locations; creation of a map of dispersal to the differentstorage locations (wherein the map is secured and encrypted and stored);and reconstructing the documents or data content. The reconstructionutilizes the map of dispersed and distributed storage and requires thepresentation of security clearances such as passwords, biometricinformation and/or physical identifiers for access at the storage leveland potentially at all the other data storage sites. The data iscompartmentalized through distributed storage and sometimes requiresseparate security clearance. This need for presenting additionalsecurity clearance at different storage locations (DS1-DS4) creates adigital bureaucratic process which enhances the security level of theentire system. The selection and extraction of data and dispersal ofthat data to select storage locations can be established under differentcriteria. For example, one level of criteria extracts last name, addressand social security numbers. Another criteria extracts every other line,every third word, etc. The parsing algorithm can utilize randomselection or systematic selection as long as the parsing algorithm isdocumented and utilized in reconstruct step 562. The parsing algorithmmay be stored with map and map store 558 or may be stored separately. Anadditional feature, as discussed above, involves utilizing place holdersor adding substitute content to the remainder data of the parseddocument 100. The use of place holders and substitute content may bethought of as an algorithm for the parsing. By using place holders andsubstitute data, private or highly confidential data is masked insuringprivacy, security, and confidentiality. The ability to parse theinformation and/or extract security information is important forfinancial transactions. The transactions which require account numbers(see Financial Document Table above) are useless without the accountnumbers. The security of the account numbers, whether identified andextracted or severely parsed and segmented, stored and reconstitutedunder security clearances, is enhanced by the present system.

To achieve a very high level of security, the system can optionallyincorporate a two-man key system. The system automatically separates theselected data stream into one or more data groups and extracts one ormore of these data groups and disperses them into data storage DS1-DS4.To release the extracted data groups and/or critical content, thereconstruct step 562 may require two persons submitting identificationcredentials or security clearances. This two-man key method is a furtherprotection against identity theft and insider attacks. The two-men keysystem can be implemented on a regular basis or on an emergency basiswhen there is need for a higher level of security.

Financial documents sometimes include substantial amounts of numericaldata such as financial projections, balance sheets, electronic fundstransfer messages, etc. It should be noted that the extraction may bebased upon a particular item such a digit and a nine digit numberrepresenting money or may be parsed automatically based upon someparsing facility. Of course, the financial document may also be viewedas a data stream with delimiters “;” separating fields in the datastream. The parsing algorithm may work on the data in each field as wellas different fields in the entire data stream.

Most storage facility systems require a map in order to reconstruct theoriginal plaintext document 100. The map may be encrypted and mayrequire a secret key sharing scheme for access thereto. Further, the mapmay be a physical map (a printout) or may be stored on a removable datastorage medium, rather than be an electronic representation. In someinstances, a map is not necessary. For example, if the security data orthe parsed or segmented data were automatically stored on a floppy disc,the originator of plaintext document 100 could move the floppy disc fromthe computer system thereby physically safeguarding the security data orthe segmented, parsed data. Without the disc, another person or theoriginator of plaintext document 100 could not reconstitute thedocument. The originator may deliver the floppy disc to another in orderto permit reconstitution. The same is true regarding removable tapes andCD-ROMs.

Advantages of the present parsing system, methodology and program,include the ability to connect to unsecured networks without adverselyaffecting the overall security of the plaintext document 100; lessdependence on existing security system including fire walls; thereduction of the requirement to keep daily updates regardingvulnerabilities of the computer system originating plaintext document100; the security of plaintext document 100 is not dependent upon thenumber of access points into the network or number of users located onthe network originating plaintext document 100; there is no damage tothe parsed and stored backup version of plaintext document 100 if newsecurity systems are installed wrong or misconfigured and there is nodamage if system administrators turn OFF the existing security systemsor improperly install or operate the security systems.

The parsing system can operate as a main security operation or anemergency backup system or as a customary backup system. The plaintextsource document or data object may be preserved with or withoutencryption, or destroyed as a further data security step. The parsingand dispersement of data protects plaintext document 100 and insures thesurvivability of plaintext document 100 if the system originatingplaintext document 100 comes under significant electronic or physicalattack. That is, if customary data storage 552 is destroyedelectronically or physically, the survivability of data in the plaintextdocument 100 is established by the present system. The storage ofgranular data groups most likely would defeat any attempt to view theentire content of plaintext document 100. Only verified user users witha confirmed security clearances or identifications verified atreconstruct step 562 and in data storage sites DS1-DS4 are permitted toreconstruct plaintext document 100. Further, the parsing of the systemcan be triggered based upon an electronic attack, an electronic hack ora physical environmental detection scheme. This system immediatelyprotects of the critical data plaintext document 100 with a transparent,automatic parsing, dispersal and storage system.

It should be noted that various aspects of the methodology and programdescribed above in connection with FIGS. 4 through 9 and 30 through 40can be incorporated into the parsing methodology and program in order toenhance or modify the system.

D.12 ADAPTIVE DATA SECURITY—EVENT RESPONSIVE SYSTEM

The present invention can also be configured as an adaptive securityprogram which adapts and adjusts the security provisions based uponintrusion into a particular network or attempts to electronically attackor hack into that network or successful hack events. Programs areavailable to track electronic attacks or hacking attempts. One of theseprograms is manufactured by Cisco and identified as the Cisco IntrusionDetection System (IDS). The Cisco IDS system can work on a server or onPCs in a network. The Cisco IDS is an electronic intrusion detector, oran electronic attack detector or a hacking monitor. The hack or attackmonitor is software loaded into a designated computer.

The output of the electronic attack or hacking monitor loaded into PC142 (FIG. 6 ) for example, or loaded into PC-6 acting as a server forNetwork A 404 in FIG. 30 , generates a plurality of attack warnings. Theattack warnings progressively and incrementally indicate the severityand degree of intrusion and hacking attacks directed to the computersystem. The following Security Level Table illustrates an example ofvarious responses to increasing levels of attacks. These increasingsecurity responses include engaging the filter and extracting criticaldata and storing it locally; the next level involves storing thecritical data on removable storage media; the next higher level involvesoffsite storage of all security data; the subsequent security alertresults in multiple offsite storage for multiple levels of security orcritical data and the highest level involves offsite storage of bothcommon data (remainder data) and security data. Of course, othercombinations responsive to the hack attack may be provided. Theelectronic attack monitor may use artificial intelligence AI to (a)assess the severity of the attack, (b) plan an appropriate “secure data”response, (c) select the degree of filter, extraction and/or encryption,and (d) locate secure extract data storage sites. AI or inferencemachines can ascertain (a) traffic on communications channels, bothintra and inter network, (b) storage limit issues, (c) transmissionfailures in the communications links, and (d) the degree of securitynecessitated by exterior events, i.e., terrorism alerts, virus alerts,war, data security warnings posted by trusted sources, Microsoft,Norton, NASA, DoD, CDC, FBI, etc. Higher security alerts trigger the AIsecurity monitor to heighten the security level (or to decrease thatsecurity level in view of a reduction or withdrawal of an electronicattack). Aspects of AI systems, inference engines and neural networksare discussed above in conjunction with the AI configured filter. TheseAI aspects can be utilized with an AI configured security sensor.

Security Level Table Attack (low threat level) Level One engage filterlocal storage - disk drive encrypt map Attack (moderate threat level)Level Two same as Level One but use removable storage media (local)Attack (nominal attack) Level Three Engage higher level filter Off sitestorage, single storage for all security data Attack (moderate attack)Level Four Multiple off site storage, multiple levels of security dataAttack (severe attack) Level Five Off site storage both common data andsecurity data

Hence, the filtering of data is based upon respective ones of theplurality of attack or hack warnings and the extraction of data anddegree of extraction is dependent upon respective ones of the pluralityof attack—hack warnings. Storage of the extracted data and the remainderdata is also based upon the degree of attack which is reflected in theattack—hack warning issued by the monitor.

FIGS. 33 a and 33 b diagrammatically illustrates a flow chart showingthe key components of the adaptive security program adaptable to variouslevels of hacker of electronic attacks. Step 460 senses all intrusionsand attempts, that is, electronic attacks, hack attacks or hackingactions on a computer or a computer network. This step is equivalent tothe output of the attack—hack monitor. Step 462 assesses the currentnetwork performance, adjusts the storage location for the extract data(the location of the extract store), the encryption level (the degree ofencryption) and the storage of the map showing the extract data storage(if necessary) and storage of remainder data, if necessary, given theseverity of the attack. For example, during high utilization of thecomputer network (high utilization in a server computer in aserver-client environment), local storage of extracted data may bepreferable as compared with offsite storage of critical data. However,if the attack occurs during non-working hours, the performance of thenetwork is very high, and the security system could utilize all theresources in the computer network to achieve the security goal of safeguarding the data during the attack. System resources include processingresources (for encryption/decryption), bandwidth resources to storeextract data and any other resources that are critical for theutilization of the security system described herein. Decision step 464determines whether a threat or attack as occurred. If not, the systemtakes the NO branch returns to step 460. If YES, the system in step 466assigns an attack level or a hack warning level to the threat or attack.The system in decision step 468, monitors the network during the attack.If the network performance or the computer performance does not change,the YES branch is taken. If the computer performance or networkperformance changes based upon or during the attack, the NO branch istaken and the system returns to step 466 which reassigns an attack levelor a warning level to the next higher or significantly higher warninglevels.

After decision step 468, the system executes step 470 which assigns thesecurity level and implements the security program based upon theattack. It should be noted that the administrator establishes the degreeof security level, the encryption, the extract store and remainder store(if necessary) for various levels of attacks or hack warnings. Thesecurity level assigned to a particular attack warning is implemented instep 470. Decision step 472 determines whether the security program'scommunication path is clear. For offsite storage of extract and/orremainder data, a communication path is important. If the path isblocked or compromised by the attack, the NO branch is taken and thesystem in step 473 reassigns the security level to a next higher levelor a different, safer security level and returns to step 470. If thesecurity and communications path is clear, the YES branch is taken fromdecision step 472 and, in step 474, the system maintains the securityprogram. Decision step 476 determines whether sufficient time has passedfrom the attack. If not, the system loops to step 474. If YES, thesystem executes step 478 which either permits reconstruction of the useroperating the plain text or source document or automaticallyreconstructs those documents that were filtered, parsed, extracted, andsubject to outside storage. The system ends in step 480. To provideadditional security, the attack monitor can be configured to monitorsecurity warnings from trusted parties such as Microsoft, Norton, NASA,DoD, CDC, FBI, etc. Emails or electronic communications from trustedparties can trigger higher levels of security. The attack monitordescribed above can be configured to accept messages from trustedparties. These messages are equivalent to detecting an electronicattack.

Further, the attack—hack monitor can be configured to monitor and assessother environmental conditions such as fire, power failure, equipmentfailure, unauthorized physical entry into the building, plant, orcomputer room. These exterior threats or events are monitored by theattack monitor since they may quickly develop into an electronic attackon the secured data retained by the computer system. In response tothese exterior events, the attack monitor generates corresponding attackwarnings similar in nature to the hack attack warnings discussed above.

There are various methodologies that may be utilized in the adaptivesystem. The tables in FIG. 34 (Standard Automatic Defenses MatrixTable), 35 (Optional Automatic Defenses Matrix Table), 36 (SecurityMeter Module Table), 37 (Normal Work Mode Table), 38 (Threat MadeTable), and 39 (Attack Mode Table) set forth these various securitymethodologies.

Another object of the system and method is to enhance the survivabilityof a system, network, or an organization through distribution ofcritical information. The objective is to enable a network ororganization to carry on its critical missions even while under attackedor damaged. Survivability is the ability of a system to execute itsmission and provide critical operational services during and after asuccessful intrusion or damage. Providing critical operational servicesincludes maintaining availability of information and data such as creditcard numbers, names, phone numbers, transaction amounts, shipmentdetails without compromising the security of the information and data.

The invention is designed to enable a network to adapt to ongoing attackand react in a way that permits critical missions to continue. With thecurrent state of the art, when firewalls or other security measures arecompromised, no real obstacles curtail or hinder intruders. The systemand method is very adaptable and flexible to provide additional layersof security, privacy, anonymity, redundancy, and backup through theselection, extraction, storage, transportation, and reconstructionprocesses. The dynamic architecture of the invention enables it toconduct an automatic real time configuration of itsextraction/transport/recovery activities, in response to the challengeof attacks.

The invention's survivability modes enable: (a) Presetting of rules forcomputer or network functioning under attack or alert. (b) An automaticassessment of damage and automatic reaction to enable functionality ofcritical missions.

D.13 EMAIL, WEB-BASED AND OTHER TYPES OF APPLICATIONS

FIGS. 42 and 43 diagrammatically illustrate a flowchart showing the keycomponents of one embodiment of the present invention, that is, ane-mail security system. A similar system may be deployed to captureselect content Sc data in real time FIG. 42 is linked to FIG. 43 viajump points 11-A and 11-B. The method of securing e-mail data operateson a distributed computer system which at least includes a remote memorydesignated as an extract store. Of course, the extract store maycomprise a plurality of extract stores operative in conjunction with aplurality of security clearance levels. The extract stores may be SCstores. A singular security or SC relevancy level is identified in FIG.42 . Further, the e-mail may be subject to a parsing algorithm which, asdiscussed above, is generally independent of the identification ofsecurity sensitive (or select content) data. However, with respect tothe parsing aspect of the present invention, the original e-mail data issplit into extracted data and remainder data and the extracted data isstored in an extract store. Hence, the parsing algorithm operatesessentially independent of the content whereas the secured e-mailprogram operates based upon content identification. Although FIGS. 42and 43 primarily relate to identification of security or SC data, thesame is true regarding the use of securing e-mail data with a parsingalgorithm.

The e-mail security or SC system begins with step 602 wherein the systemor program is turned ON or is activated. Step 603 recognizes that theuser originating plaintext document 100 (not shown) has set a securityor SC filter identifying one or more security sensitive (or selectcontent SC relevant) words, characters or icons. In step 604, the usercomposes the e-mail representative of plaintext document 100. In step606, the user selects the “send” command in the typical e-mail program.As is customary, the system in step 608 conducts a spell checkingroutine prior to sending the e-mail. In step 610, the system conducts asecurity or SC check on the plaintext document or composed e-mailgenerated in step 604. The filter is used in step 604. In step 612,security or SC words are highlighted or distinguished in the e-mailprior to the actual sending of the e-mail to the addressee. This step612 is optional. In step 614, the user selects the security or SC wordsfor data to be extracted out. The highlighting step facilitates thisselection. In step 616, the system extracts the security or SC data and,preferably, in step 618, the security or SC data is encrypted. Step 618is optional. In a parsing application to secure e-mail, the parsingalgorithm operates automatically at step 610 thereby eliminating steps612 and 614. The extracting step 616 simply represents that thesegmented data obtained from the original plaintext e-mail generated atstep 604 is separated from remainder data.

After encryption step 618, the e-mail security or SC system generallyoperates in one of three manners. Other systems may be formulated basedupon the systems and subsystems discussed herein. In one methodology, asecond e-mail is created (see step 629), in a second methodology thesecured or SC data in encrypted form is attached or appended to theoriginal e-mail containing remainder data (step 621) or, in a thirdmethodology, the encrypted security or SC data is simply added to orinserted into the end of the remainder data of the e-mail (step 623).The methodology of generating a second e-mail is initially discussed.

A second e-mail having encrypted security or SC data is created in step620. Further, the system in step 622 adds a hyperlink to the remainderdata in the original e-mail created in step 604. The hyperlink presentsa pointer for the addressee to a secured application service providerASP or SC application service provider ASP. See the discussion of FIG. 6above. The ASP represents a data storage facility for the secured or SCe-mail data. In step 624, the remainder data from the original e-mail issent to the addressee in a normal manner. This step also includes theconcept that the second e-mail containing the encrypted security or SCdata is sent to the ASP. In step 626, the addressee receives theremainder e-mail which includes a hyperlink to the secured or SC dataASP. The system jumps at jump step D-11-A from FIG. 42 to FIG. 43 .

In step 628, the addressee receives the remainder e-mail, visits the ASPvia the hyperlink and clears the security levels at the secured ASP. Inthe SC system, the SC data is stored in the SC storage and this datacapture is logged into the system. In step 630, the secured or SC dataASP obtains a map for each secured or SC data e-mail (since the originale-mail may be broken up into a plurality of extracted, secured or SCdata e-mails) obtains all secured or SC data e-mail and decrypts thesame. In step 632, the secured or SC ASP downloads the secured or SCdata as an e-mail to the addressee. In step 634, the addressee systemcompiles the original plaintext e-mail 100. In the SC mode, the SC datain added to the remainder data and the user-viewer is permitted to seeadditional SC content associated with that relevancy level. In thissense, the SC email is a knowledge expander permitted the user-viewer toenhance his or her knowledge with additional SC content. Areconstruction program may be necessary to decode the secured or SC dataand insert the data into the document via the placeholders.

Optionally, the decryption could occur at the recipient's e-mail devicesomewhat prior to the reconstitution of the e-mail plaintext document100 during step 634. This requires the addressee to have the encryptionroutine and the correct key or decrypt code. The e-mail security or SCsystem described above may include many of the features discussedearlier in connection with the other figures. For example, both thesecurity or SC data and the remainder e-mail data can be encrypted priorto transmission to the addressee and the secured or SC data ASP. Theencryption may include multiple levels of encryption and decryption mayrequire multiple levels of security clearance. The encryption may bemixed in the remainder e-mail. Partial as well as full reconstruction isenabled as discussed above in connection with FIG. 7 .

From the senders or originator's viewpoint, the e-mail facilitydescribed herein facilitates the storage of the extracted data at one ormore secured or SC sites.

Another implementation of the secured or SC e-mail system attaches theencrypted and secured or SC data to the remainder e-mail data asindicated in step 621. For an SC email system, the addressee may add theSC sent with the email to the addressee's SC database. E-mailattachments are well known. Alternatively, the encrypted secured or SCdata may be embedded or copied in encrypted form at the end of theremainder data in the original e-mail as indicated in step 623. Ineither case, in step 625, the e-mail is sent to the addressee. In step627, the addressee opens the attachment. In step 629, the system of therecipient decrypts the secured or SC data attachment or the embeddeddata attachment. In step 631, the recipient's system integrates the nowdecrypted secured or SC data with the remainder data. Of course, this acompilation step. Place holders or other position indicators arecustomarily utilized. Appending the encrypted security or SC data isgenerally equivalent to attaching a file to the original e-mail whichconstitutes, after extraction, the remainder data. Including theencrypted security or SC data is adding the security or SC data to theoriginal e-mail at a predetermined location (either the top of thee-mail, the bottom of the e-mail or some predetermined line number). AllSC data may be added to the addressee's SC store. Relevancy of the SC isbased upon SC relevancy markers transmitted with the email, similar tothe security levels for the MLS secure email system.

It should be appreciated that the e-mail or SC security system may workautomatically or may be selected manually by the user. The highlightingor special distinguishing manner for the security or SC words in step612 is optional. By highlighting the security or SC words, the user mayselect or deselect those words for extraction. At the addressee's side,the addressee's system may be configured to automatically seek out thesecured or SC data ASP, enter security clearance data, download thesecure or SC data and integrate the secure or SC data in the remainderdata e-mail (or SC data structure). The present invention contemplatesautomatic as well as manual steps in steps 626, 628, 630, 632 and 634.The hyperlink with the original remainder e-mail essentially maps theremainder data to the secured or SC data and the remote storagelocations handling the secure or SC data. Multiple security clearancesmay be required of the recipient or addressee. The e-mail system can becombined with other features of the security or SC system discussedabove such as multiple security data locations, secret key sharingschemes, multiple encryption of the data in a single document, multiplesecurity clearance levels required for a plurality of storagefacilities, the two man key system, automation of key management and aplurality of levels of access to the data such as partial reconstructionin step 634 and full reconstruction.

FIGS. 44 a and 44 b diagrammatically illustrate a flowchart showing thekey components of one embodiment of the system and the invention whichimplements the security or SC system on a web browser. Jump point 12-Alinks FIG. 44 a to FIG. 44 b . The system, at step 700 is ON. Thefilters establishing either the parsing or the identification ofsecurity or SC data are established in the filter set step 701. In step702, the user inputs data into open field of an HTML display page whichthe user has previously downloaded from a web server. In step 704, theuser may select “secure now” turning ON the system (or ON SC operator)or the system may automatically be ON such that the filter is screeningall the data input by the user in the open field. In step 706, thesystem scans all the open field data, locates security or SC data andextracts security or SC data. In step 708, place holders are added toreplace the extracted security or SC data in the remainder data and ahyperlink is added to the open field remainder data providing a link tothe secure or SC data ASP. In the SC system, the placeholders show theaddressee critical terms subject to the SC knowledge expander feature.In step 710, the user selects the “send button” or any other indicatoron the HTML page triggering an operation which transmits the open fielddata (which is now remainder data) to the web server. In step 712, theweb server and particularly the common gateway interface (CGI) receivesthe remainder data fields, identifies the place holders in the data andthe hyperlink to the secure or SC data ASP. In step 714, the web serverreceiving the data from user's browser goes to the secure or SC dataASP, inputs and clears any security level, and obtains the secured or SCdata. In step 716, the web server reconstructs the open field data whichgenerally is represented by plaintext document 100. In step 718, the webserver processes the data as necessary. Many of the features discussedabove in connection with the above figures may be implemented on thebrowser system.

The credit card scrubber or financial data scrubber operates in asimilar manner to the email and browser data security system describedabove. A similar system can be designed to comply with privacy laws orother governmental regulations. A credit card number is similar to asocial security number and patent name, address and phone are commonpersonal identifiers that some privacy laws require extraction andspecial handling. The credit card or financial data or social securitynumber scrubber (herein collectively “CC scrubber”) typically operateson a defined sequence of numbers. For example, if a credit card numberis 17 digits, whenever the email or browser security system or programdetects 17 sequential numerical digits (a pre-set filter), a pop-upwindow may appear enabling the user to select or turn ON the scrubber.If ON, the data security program strips or parses the credit card numberand sends, for example, five of the 17 digits to a secure store. In anSC system, the CC data may be data mined by affiliated companies.Placeholders or substitute characters may be inserted into the remainderCC data. To reconstitute the entire CC data, the intended recipientwould be required to pass security clearance levels at the secure store.An SC system requires SC relevancy clearances set by the policyadministrator. Of course, the CC scrubber could be set to detect bankaccount numbers, personal or business account holder names, pre-setpasswords, etc. In an OFF state, the CC scrubber would let pass the CCnumber, account number or pre-set data stream or string. The user mayselect (i) always ON; (ii) pop-up window, select ON or OFF pertransaction; (iii) pop-up window to select OFF (default being ON); or(iv) always OFF but minor reminder (audible sound, icon appearance,etc.) of data security risk. The CC scrubber may encrypt the extracteddata for security. Other visual ques may rather than a pop-up window maybe used (for example, a drop down menu). The scrubber can also bedeployed on wireless devices to scrub sensitive data such as credit cardand other financial data.

FIG. 45 diagrammatically shows several revenue systems which may beemployed with the data security systems described herein. Many types ofrevenue systems may be employed in conjunction with the presentinvention. FIG. 45 shows two basic systems, one at the data input stageand the second at the data output or reconstruction phase. Release ofthe reconstructed document or portions thereof are based upon securityclearance or SC relevancy and compensation. “Compensation” includes anexchange of information, particularly for the SC storage system. Withineach revenue subsystem are two types of revenue generators, anadvertising revenue generator and a user charge generator. The usercharge system contemplates charging or assessing a fee to the user'semployer or organization. Therefore, the system operator may select upto four (4) revenue generation systems (ads at the input, charges at theinput, ads at the output and charges at the output). It is well knownthat vendors selling goods and services over the Internet are willing topay a certain percentage of their sales revenue to other entitiesreferring customers to the vendor's web sites. The concept of displayads in FIG. 45 includes this revenue stream. The system operator maychoose all, one, several or none of these revenue systems to be deployedin conjunction with the data security system described earlier herein.Other revenue system may also be utilized. The steps in the revenuesystem described herein may be reorganized to attain higher consumer anduser acceptance and/or to maximize the revenue to the system operator.Swapping SC content may include an evaluator module that tests the“proposed” SC data file and determines whether the proposed SC data fileis worthy or being exchanged for an existing SC data file.

Decision step 730 determines whether the system is deployed at the datainput phase or not. It is clear that the system operator may utilize thedata reconstruction revenue system and hence the decision step 730 isnot necessary. If the data input system is employed, step 732 displaysthe ad to the user. The user may be uploading a complete document to anapplication server on the Internet or may be using an applicationservice provider on the Internet or an private LAN to secure his or herdata. The display ad 732 step enables the user to click on the ad andvisit the vendor, thereby potentially generating a referral fee. Seereferral fee branch 757. Step 734 requires password clearance. Step 736processes the document or data object with the security or SC system.The user may input the document real time or input it to the applicationserver or may upload the complete document to the server. Alternatively,the ad could be buried in the email or application program run on theuser's computer and the user would be shown an ad and given a link tothe vendor's Internet site. Selecting the link points the user's browserto the vendor's site.

Step 738 shows display ad 2 to the user thereby potentially generatingreferral revenue for the system operator. Step 740 notes that the userexits the revenue system. Step 742 determines whether the system chargesthe user for the security or SC service. If YES, the program processesthe charge in step 745 (charge systems are known). If NO, the systemends or returns to other programs in step 747.

The NO branch from determination step 730 leads to the receipt of areconstruction request by the user in step 750. Step 752 determineswhether the user will be charged. If YES, the system executes step 745.If NO, the system displays the ad 1 in step 754. Referral generation isnoted by branch 757 from step 754. In step 756, the user's password issubject to clearance. In step 758, the user's request is processed, thedocument or data object is reconstructed (fully or partially asdescribed earlier), and in step 759 the system displays ad 2. In step762, the user's activity is logged in to the system. Step 764 determineswhether the charge to the user is reduced (because he or she viewed theads) and if not, the system ends in step 747, if YES, the systemprocesses the charge in step 745. Alternatively, the user may be showndisplay ads and/or charged for services upon storage of extracted data.Step 750 includes this concept.

D.14 PORTABLE COMPUTING DEVICE ENVIRONMENT

The invention can be applied to portable computing devices to securefiles and data objects in such devices. The invention extracts,disperses, via a controlled release of data segments to storagelocations, and permits reconstruction utilizing security protocols toprovide a security or SC system for data based upon the location of theportable device, typically detected by a global position signalgenerator (GPS) or based upon triangulation data from several broadcastpoints. In an SC system, the sensitive data may be made available to theuser-viewer for a period of time or as long as the user is in ageographic territory. Beyond that time-space limit, the SC data isremoved from the user's portable data device. Scrubbing security or SCicons from maps, credit card data or financial data from text, a dataobject or data stream is part of the portable security system.

As used herein, the term “portable computing device” means a laptopcomputer, a PC with a movable feature, such as a PC mounted in a car,plane, truck or trailer, PDAs or personal data assistants, mobile orcellular phones configured with a memory, a processor and some type ofGPS or locator system to determine where the phone or cellular unit islocated within a territory and digital pagers having similar electronicsystems.

The present invention can be linked with a location sensing circuit,such as a global position sensor or system (GPS) or other type oflocation sensing system, such as a system which utilizes triangulatedsignals. The concept is a location based access oriented security (or SCrelevancy) such as an automated trigger (which activates the security orSC program discussed hereinabove when the portable computing device isbeyond a predetermined region); an automated safety system; a trip wire;an interlock; a method to disable systems, activity or access to data;and means to limit functionality or access in whole or in granularparts. The portable security system operates on text, data objects,images or other digitally configured data objects. Security access (orrelevancy access) is limited by a location way point (in relation to areference point) or a calculated range (using satellite GPS, highaltitude services, or earth-based range finding GLS (geographic locationservices)) about a way point with physical means or mathematicalcalculations to define a geographic area by equations or geometricshapes or aggregated ranges (the shapes including rectangles, solids,cubes, circles, oval, spherical region or other areas defined byalgorithms). Physical and logical access or entry control to weapons,devices, vehicles, computers, equipment, tools, data, networks, localaccess, remote access beyond a physical location (reference point), canbe enabled or disabled with the system of the present invention. Theregions (sometimes identified as a single predetermined region or aplurality of predetermined regions), can consist of complex definitionsof three-dimensional areas of arbitrary shape and sizes, as long asthose regions can be defined by algorithms. The region can also bedefined as an area circumscribed internally by a perimeter or by an areaexternal to that perimeter. In other words, access can be denied if theportable device is within a certain region as compared with denyingaccess when the device is beyond a predetermined region. The claims aremeant to cover both situations.

FIG. 46 diagrammatically illustrates a portable computing device 810 atlocation B. The portable computing device 810 includes, in theillustrated embodiment, a GPS system (or a receiver system) 812 coupledto a bus 814 and further coupled to memory 816, a processor 818 and aninput/output system 820. Input/output 820 is coupled to, among otherthings, a key board or key pad, a display, and possibly a transmitterand receiver subsystem. As is known, GPS Systems detect satellitepositioning signals and generate an output indicative of the location ofthe GPS system. In the illustrated embodiment, this location is locationB in FIG. 46 .

A simple implementation of the present security or SC system providesthat upon detection of d1 from location A, defined by building 822,certain security events occur, e.g., automatic extraction and a denialof reconstruction rights. In one example, GPS subsystem 812 continuallymonitors the location of portable device 810. When the location ofdevice 810 exceeds a predetermined distance (d1-limit), the programoperating in memory 816, operable by processor 818, either extracts dataand stores the extracted data as discussed in detail above or prohibitsreconstruction of data as requested by the operator of portable device810. Alternatively, automatic extraction may occur without prohibitingreconstruction due to device 810 being located beyond the predeterminedregion d1-limit. The portable computing device 810 in FIG. 46 may havemany other electronic components such as those shown in FIG. 6 inconnection with computer 165. Alternatively, the security or SC systemcan be configured in a reverse manner such that the extraction ofsecurity or SC information is triggered when portable 810 is within apredetermined region (less than d1-max) close to location A and building822, that is, the security or SC system disclosed above is triggered toextract information when distance d1 is less than d1-max.

The security or SC system can also be configured such that GPS orlocator system 812 detects a variable distance such as distance d2between truck 824 and location B of portable device 810. In this sense,the location of portable device 810 is obtained by GPS circuit 812 andfurther some type of communications must be established between truck824 at location C and the portable device 810. For example, the receivercoupled to input/output 820 receives this information from location oftruck 824 and location C. This reference location C is then processed inconjunction with the location data from GPS circuit 812 by processor 818and memory 816. The same results as discussed above in conjunction withfixed reference location A can be achieved with a variable referencelocation C. Truck 826 at variable location D enables the system toprovide an additional level of security. In other words, within distanced2 (d2-limit), the operator of portable device 810 may be able toreconstruct information upon request. However, if portable device 810intrudes upon or is less than distance d3 (d3-max) the distance betweenlocation B and location D, the security system may trigger an immediateextraction routine thereby disbursing, on a granular basis, the securedor SC words, data objects or whatever and further prohibitreconstruction. Of course, the security or SC system could be configuredsimply to extract the information and permit reconstruction. Otherwise,the security or SC system could be configured to simply extractinformation and prohibit reconstruction. In this manner, the security orSC system discussed in conjunction with the portable computing device810 can have multiple triggers or location established events enablingthe security or SC program to extract security or SC information ordisabling the security or SC program to prohibit reconstruction basedupon a fixed location A or one or more variable locations C, D.

Another configuration of the present invention utilizes triangulation toobtain location B for the portable computing device 810. In atriangulation situation, the receiver system 812 for the locatorreceives signals from one or more fixed locations, and preferably threelocations diagrammatically illustrated by tower T1, T2 and T3 in FIG. 46. The triangulation of signals to obtain location B is known in the art.However, the combination of such location position coupled to extractionof security or SC data and/or reconstruction of security or SC databased upon certain location parameters is part of the present invention.

For illustration purposes only, the present security or SC system forthe portable computing device 810 can remotely store extracted securityor SC information. Remote store 828 coupled to receiving tower T1illustrates this concept.

FIG. 47 diagrammatically illustrates a basic flow chart for the portablesecurity or SC system program utilized in connection with portablecomputing device 810. Decision step 830 determines whether a certainevent has occurred. This event may include a power ON for the portablecomputing device, may be a “save document” command, may be a screen ONevent or may be a timed function. For example, if the portable computingdevice 810 is continually ON, the program may periodically poll thelocator circuit (GPS 812) and determine whether location B is within orbeyond the predetermined regions (d-max or d-limit). Step 832 activatesthe GPS or the triangulation circuit to obtain current location data.This may include locating variable locations C, D. Step 833 obtainsterritory limits for various security levels or SC relevancy levels. Asdiscussed earlier, security or SC level SL1 is public ornon-confidential information, security or SC SL2 confidential orproprietary information, level SL3 is secret information and level SL4is top secret or SC information. This system can be configured such thatvarious territories or predetermined regions correspond to respectiveones of the security or SC levels SL1-SL4.

Decision step 834 determines whether location B of portable computingdevice 810 is within or without the predetermined territory limits orpredetermined region. If YES, the system determines in decision step 836whether the user has initiated a reconstruction request. If not, thesystem returns to a point preceding decision step 830, the detect eventfunction or step. If YES, the system, in step 838, reconstructs thedocument only if location B is beyond a predetermined region. Of course,a negative operation could occur in that reconstruction would beprohibited if location B was within a predetermined region. The claimsappended hereto are meant to cover both within a region and without aregion and independently, extract or permit reconstruction. Step 840secures the reconstructed file again as necessary.

Returning to decision step 834, if location B is not within thepredetermined regions or territories defined by security or SC levelsSL1-SL4, the NO branch is taken and decision step 842 determines whetherportable computing device 810 has any unsecured files. If YES, thesystem executes step 844 which is extract and store the security or SCsensitive words, data objects etc. in accordance with security or SClevels SL2-SL4. As stated above, the storage could be on media in alocal drive or can be remotely distributed to memory segments designatedas a remote extract store. If the NO branch is taken from decision step842, the system executes decision step 846 which determines whether theuser has requested a reconstruction of data. If not, the program ends orreturns to the event detection step 830. If YES, the system executesstep 848 which determines whether a lower security or SC clearance isavailable within the current territory, determines whether the user hasthe proper pass code to access the reconstruction and process thereconstruction or whether the system prohibits all reconstruction.Partial reconstruction for lower security or SC items may be permitted.For example, reconstruction at top secret or SC level SL4 may beprohibited when distance d2 is greater than d2-limit but reconstructionat a lower security or SC level such as confidential level SL2 may bepermitted beyond limit d2-limit. In this sense, the present inventioncan be configured to generate extraction for various security or SClevels at various predetermined regions based upon a fixed referencepoint or a variable reference point. Alternatively, reconstruction canbe permitted or denied based on a plurality of security or SC levels anda plurality of corresponding regions or distances. The term “mobilepredetermined region” is sometimes utilized in conjunction with variableregions d2 and d3.

As an example, the data object retained by portable computing device 810may be a map having security or SC sensitive icons on the map. Theseicons are extracted if location B is less than a predetermined defineddistance d3-limit between variable location D and location B. Iflocation B is beyond d3-minimum, the map can be viewed by the operatoron portable device 810. If location B is less than distance d3-minimum,the security or SC icons are removed from the map. In a similar sense,security or SC sensitive credit card characters can be extracted fromplain text documents carried on portable computing device 810 whendevice 810 is a certain distance beyond d1-limit from fixed referencepoint A. This location A may be a bank headquarters. Encrypting anddecrypting the data based upon the geographic event is also contemplatedby the present invention. Of course, portable device 810 may be aplurality of portable devices linked via a hard wire network or via awireless network. The same security or SC program disclosed above inherein can be utilized with one computer or a series of computers.Further, portable computing device 810 can include a plurality of memorysegments (see FIG. 8 ) and may include a plurality of display screens asdiscussed above in conjunction with FIG. 7 . The extraction and storageand reconstruction of streaming data is possible as is operation onvoice data. Additionally, the portable computing device may set off anaudible and/or visual alarm prior to extraction of data. For variableterritories or predetermined regions, step 832 or 833 may includegathering information regarding the variable location of vehicles 824,826 prior to determining the territorial limits for various security orSC levels SL2, SL3 and SL4.

E.1 GENERAL SYSTEM COMMENTS

It is important to know that the embodiments illustrated herein anddescribed herein are only examples of the many advantageous uses of theinnovative teachings set forth herein. In general, statements made inthe specification of the present application do not necessarily limitany of the various claimed inventions. Moreover, some statements mayapply to some inventive features but not to others. In general, unlessotherwise indicated, singular elements may be in the plural and viceversa with no loss of generality. In the drawings, like numerals referto like parts or features throughout the several views. The sectiontitles are not meant to limit the detailed description of the system andprocess described therein.

The present invention could be produced in hardware or software, or in acombination of hardware and software, and these implementations would beknown to one of ordinary skill in the art. The system, or method,according to the inventive principles as disclosed in connection withthe preferred embodiment, may be produced in a single computer systemhaving separate elements or means for performing the individualfunctions or steps described or claimed or one or more elements or meanscombining the performance of any of the functions or steps disclosed orclaimed, or may be arranged in a distributed computer system,interconnected by any suitable means as would be known by one ofordinary skill in the art.

According to the inventive principles as disclosed in connection withthe preferred embodiment, the invention and the inventive principles arenot limited to any particular kind of computer system but may be usedwith any general purpose computer, as would be known to one of ordinaryskill in the art, arranged to perform the functions described and themethod steps described. The operations of such a computer, as describedabove, may be according to a computer program contained on a medium foruse in the operation or control of the computer as would be known to oneof ordinary skill in the art. The computer medium which may be used tohold or contain the computer program product, may be a fixture of thecomputer such as an embedded memory or may be on a transportable mediumsuch as a disk, as would be known to one of ordinary skill in the art.

The invention is not limited to any particular computer program or logicor language, or instruction but may be practiced with any such suitableprogram, logic or language, or instructions as would be known to one ofordinary skill in the art. Without limiting the principles of thedisclosed invention any such computing system can include, inter alia,at least a computer readable medium allowing a computer to read data,instructions, messages or message packets, and other computer readableinformation from the computer readable medium. The computer readablemedium may include non-volatile memory, such as ROM, flash memory,floppy disk, disk drive memory, CD-ROM, and other permanent storage.Additionally, a computer readable medium may include, for example,volatile storage such as RAM, buffers, cache memory, and networkcircuits. Furthermore, the computer readable medium may include computerreadable information in a transitory state medium such as a network linkand/or a network interface, including a wired network or a wirelessnetwork, that allow a computer to read such computer readableinformation.

In the drawings, and sometimes in the specification, reference is madeto certain abbreviations. The following Abbreviations Table provides acorrespondence between the abbreviations and the item or feature.

Abbreviations Table A-com computer or memory store for common orremainder data ASP application service provider - server on a networkB-ext computer or memory store for extracted data bd board CD-RW compactdisk drive with read/write feature for CD disk CIO Chief InformationOfficer or system administrator comm. communications, typicallytelecommunications comp computer CPU central processing unit DB or dbdata base or structured data file D & R dispersion and re-construct orre-assemble Distrib distribution or transmission of data (information)files DS data storage doc document dr drive, e.g., computer hard drive eencryption ext-data extracted data F floppy computer drive or flashdrive memory H-tax hierarchical taxonomic system, a prioritizedclassification system I/O input/output I-com Internet storage for commonor remainder data id identify I-ext Internet storage for extracted dataKE knowledge expander, such as a knowledge expander search loc locationMLS multilevel security mem memory mod module orsoftware-firmware-hardware function obj object, for example, a dataobject opt optional PC personal computer or any other type of generalcomputer pgm program re regarding or relating to recon reconstruct relrelease req request rev review SC select content -- e.g., words, terms,images, sound that is of particular interest to the user sec securitysec-con security sensitive content sec. level TS top secret, S secret, Cclassified, UC unclassified sel SC sensitivity level, sometimes SC sel.level sel cont select content sel levels SC levels, hierarchicaltaxonomic classification levels of SC sch search SL security level(sometimes S1 for sec. Level 1, S2 is Level2, etc., also, for example,TS is Top Secret, S is Secret, C is Classified, U is Unclassified))supple supplemental supple doc supplemental document of data object syssystem sys-admin CIO or system operator t time tax taxonomic orclassification system tele-com telecommunications system or network unknunknown item or data element or data object URL Uniform ResourceLocator, x pointer, or other network locator W St computer work station

“Data,” as used herein, includes any data object, e.g., text, image,icons, audio, video, still images, etc. and data is sometimes referredto herein as a “data object.” A source document is either a document,any data structure, or a data stream. Since a data stream has a startbit or term and an end bit or term, the data stream is structured data,and reference to a “document” refers to any document, data structure, ordata stream. Likewise a “supplemental document” is any document, datastructure, or data stream. The select content (SC) is any critical orsecurity sensitive word, character, image, or data object as pre-definedor as established by the user. The user may specifically design thefilter, begin with a dictionary to define common terms, identify anyadditional SC words, letters, images, icons, partial versions of theforegoing or any other granular aspect of the source document.

FIG. 29 diagrammatically illustrates a personal computer or PC computersystem 140, a second PC or computer 142, and a third PC-3. Similarnumerals designate similar items in FIGS. 29 and 6 . PCs 140, 142 andPC-3 are connected together via a network 145 (LAN or WAN) and connectedto one or more private databases or data structures which arerepresented by db A. The LAN 145 is also connected to an input/outputdevice 146 that may be generally described as a router or an I/O deviceto an outside communications system. The input/output device 146 isconnected to a telecommunications system 148 which leads to Internet150. The Internet is a global computer network. Internet 150 is coupledto a plurality of servers, one of which is server 152. Server 152 may bedesignated as an application service processor ASP. The ASP providesvarious tools to the user on computer 142 such a Google search throughother data bases, indices and data structures. Internet 150 alsoincludes various computer memory storage devices such as computerstorage B 154 a, computer storage C 156 a accessible through Gate C 157a (via password or other control device) and computer storage D 158 aaccessible via Gate D 159 a. Access to computer storage C and D ispermitted via, in one instance, security level clearance modules or GateC and D 157 a, 159 a. Maps to the computer stores C and D may requiresecurity level clearance.

Typically, PCs include a keyboard or data input device 161, a display163, a central processing unit CPU 165, a video board 167 having videoboard memory 169, a fixed disc hard drive 168, a RAM 166, andinput/output device 164, a removable memory F media (flash or floppy)drive 162 and a removable compact disk (CD) read-write (CD-RW) device ordrive 160. The system may include other removable disk drives, tapedrives, or flash memory units. Internal units CPU 165, video board 167,hard drive 168, RAM 166 input/output device 164, floppy drive 162 andCD-ROM device 160 are all coupled together via an internal bus 171. Bus171 represents a plurality of buses as is known to persons of ordinaryskill in the art.

One methodology of implementing the present invention utilizes filterson the PC or on removable memory F. The present invention may also beembodied utilizing an Application Service Provider (ASP) on server 152and in a client-server network. The user or operator on the PC 142 callsup an ASP on system 152 and operates the KE process on the computersystem 152. Filters and rules may be located with a uniform researchlocator or URL to find filters, data collections, target files fromcomputer store B, C and D. In a client-server environment, server 152acts as a server generally cooperating with data operations with clientcomputer 140. Of course, persons of ordinary skill in the art recognizethat the server may be located on the local area network 145 rather thanbeing interconnected with Internet 150 as shown in FIG. 29 . The claimsappended hereto are meant to cover the alternative embodiments.

As an example of a client-server or web-based implementation of thepresent invention, the user at computer 140 may define the SC 2018 asdescribed above, and input data via keyboard 161 or load source datafrom F drive 162 or CD-ROM drive 160 into RAM 166. Alternatively, thedata stream on LAN 145 may be monitored by PC 140. In any event, whetherthe source data is input via keyboard 161 or copied or accessed in orfrom F drive 162 or CD-RW drive 160, the source data is filtered asdiscussed above in connection with FIGS. 11-12 a. Prior to filtering, itis appropriate for the user at computer 140 to identify the hierarchicaltaxonomic system and the location of SC filters and My Group Profiles.Offsite data storage and processes are available permitting activationof server 152 and enabling the server to take over the process directlyfrom user 140. In other words, the user at computer 140 could call upthe URL of the server 152, the server could request certain userinformation (user name, password), and would request data from theclient computer to establish the SC filter and H-tax classes pursuant toinput selected by the user. The client compute may (a) filter the sourcedoc thereat or (b) send the data to the server for filtering. The servercould store data either locally on computer 140 or remotely at computermemories 154 a, 156 a. Either the PC or the server could conduct theknowledge expander search and return the compiled data, supple docs, andmap results to the PC.

The claims appended hereto are meant to cover modifications and changeswithin the scope and spirit of the present invention.

What is claimed is:
 1. A method of data mining and processing a datathroughput with a computerized search through electronic datacollections, represented by databases or data files, collections orstructures, comprising: identifying and extracting security sensitivecontent (Sec-con) data and select content (SC) data from the datathroughput based upon one or more enterprise designated (ED) categoricalfilters from a group of ED filters including (i) a customer privacyfilter, (ii) a supplier privacy filter, (iii) an enterprise humanresource filter, (iv) an enterprise financial filter, (v) an enterprisetechnical innovation filter, and (vi) a competitive enterprise filter;storing Sec-con data in one or more Sec-con extract data stores andseparately storing SC data in SC extract data stores; from the SC datastores, extracting search terms with a content filter, a contextualfilter, and a taxonomic filter; processing the search terms through acontent inference engine; changing SC data from the SC data stores toSec-con data based upon security classifications revealed by the contentinference engine and storing the changed SC data in the Sec-con datastores; deleting the changed SC data from the SC data stores; searchingsaid data collections with first search terms and gathering firstsupplemental documents based thereon; and determining a search relevancyfactor as a relationship between said throughput data and said firstsupplemental documents based upon a comparison of data elements in saidthroughput data and the data elements in said first supplementaldocuments.
 2. The method of data mining as claimed in claim 1 including:extracting secondary search terms from said first supplemental documentswith one or more of said content filter, said contextual filter, or saidtaxonomic filter; repeating the search of said data collections withsaid secondary search terms and gathering secondary supplementaldocuments; repeating identification and extraction of Sec-con data andSC from the secondary supplemental documents.
 3. The method of datamining as claimed in claim 2 including: determining whether said firstsupplemental documents and said secondary supplemental documents exhibitconvergent or divergent characteristics.
 4. The method of data mining asclaimed in claim 1 including: identifying unknown data elements fromsaid data throughput input data; and repeating the search of said datacollections with said unknown data elements applied as additional searchterms and gathering additional supplemental documents.
 5. The method ofdata mining as claimed in claim 4 wherein repeating the search of saiddata collections gathers tertiary supplemental documents.
 6. The methodof data mining as claimed in claim 5 including: using said taxonomicfilter and said contextual filter wherein said taxonomic filter sets adata element range which range is applied to said contextual filterprior to extracting said first search terms.
 7. The method of datamining as claimed in claim 1 wherein said relevancy factor is determinedby using said content filter, said contextual filter, and said taxonomicfilter on said first supplemental documents.
 8. The method of datamining as claimed in claim 1 wherein said taxonomic filter effects aselection of one or both of said content filter and said contextualfilter.
 9. The method of data mining as claimed in claim 1 includingcontrolling said searching with the first search terms with a userselected control from the group of controls including a continuoussearch, an iterative search of n search cycles, a search with m searchterms, a search over a predetermined period of time.
 10. The method ofdata mining as claimed in claim 1 wherein: said data collections aredispersed over a communications network as dispersed data collections;and said searching of said dispersed data collections creates afederated search result with federated first supplemental documents. 11.The method of data mining as claimed in claim 10 wherein: said datacollections are designated with respective sensitivity levels accessedvia said communications network; and permitting access to respectivesensitivity level data collection via said search only in the presenceof a corresponding security clearance.